STEREOSCOPIC (3D) PANORAMA CREATION ON HANDHELD DEVICE

• US 20110141227A1
• Filed: 09/09/2010
• Published: 06/16/2011
• Est. Priority Date: 12/11/2009
• Status: Active Grant

• Current Assignee: FotoNation Limited (Adeia Inc.)
• Original Assignee: FotoNation Limited (Adeia Inc.)
• Inventors: Drimbarean, Alexandru, Bigioi, Petronel, Corcoran, Peter, Susanu, George, Barcovschi, Igor, Murray, Larry, Stec, Piotr
• Granted Patent: US 10,080,006 B2
• Time in Patent Office: Days
• US Class Current: 348/36
• CPC Class Codes: G06T 2200/32 involving image mosaicing
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 7/593 from stereo images
• CPC Class Codes: H04N 13/106 Processing image signals for multi-view video sequence encoding H04N19/597

Abstract

A technique of generating a stereoscopic panorama image includes panning a portable camera device, and acquiring multiple image frames. Multiple at least partially overlapping image frames are acquired of portions of the scene. The method involves registering the image frames, including determining displacements of the imaging device between acquisitions of image frames. Multiple panorama images are generated including joining image frames of the scene according to spatial relationships and determining stereoscopic counterpart relationships between the multiple panorama images. The multiple panorama images are processed based on the stereoscopic counterpart relationships to form a stereoscopic panorama image.

Claims

1. A method for generating a stereoscopic panorama image, the method comprising:

fixing an exposure level for acquiring the panorama image with the portable imaging device;

panning the imaging device across a scene;

acquiring multiple at least partially overlapping image frames of portions of said scene, including using an optic and imaging sensor of the portable imaging device;

registering the image frames, including determining displacements of the imaging device between acquisitions of image frames;

generating multiple panorama images including joining image frames of the scene according to spatial relationships and determining stereoscopic counterpart relationships between the multiple panorama images;

processing the multiple panorama images based on the stereoscopic counterpart relationships to form a stereoscopic panorama image; and

storing, transmitting or displaying said stereoscopic panorama image, or combinations thereof.

2. The method of claim 1, wherein the determining of displacements comprises measuring the displacements.

3. The method of claim 1, wherein the determining of displacements comprise determining displacements in at least two dimensions.

4. The method of claim 3, wherein the at least two dimensions comprise two dimensions orthogonal to the depth dimension.

5. The method of claim 4, wherein the determining of displacements comprise determining displacements in three dimensions including the depth dimension.

6. The method of claim 1, wherein the determining stereoscopic counterpart relationships comprises:

pairing image frames having relative displacements within a first predetermined range configured to provide a selected stereoscopic effect;

combining paired image frames to form component stereoscopic images; and

retaining paired image frames having overlap with adjacent paired image frames within a second predetermined range.

7. The method of claim 6, wherein the first predetermined range comprises 5-7.5 cm.

8. The method of claim 1, wherein the partially overlapping comprises between 10% and 50% overlap.

9. The method of claim 1, further comprising interleaving joining of pairs of image frames with acquiring or generating, or both, of next image frames.

10. The method of claim 1, wherein the first predetermined range is not greater than 15% of a panning radius of the imaging device.

11. A portable camera-enabled device capable of in-camera generation of a panorama image, comprising:

a lens;

an image sensor;

a processor; and

a processor readable medium having code embedded therein for programming the processor to perform a stereoscopic panorama image generation method that comprises;

fixing an exposure level for acquiring the panorama image with the portable imaging device;

panning the imaging device across a scene;

acquiring multiple at least partially overlapping image frames of portions of said scene, including using an optic and imaging sensor of the portable imaging device;

registering the image frames, including determining displacements of the imaging device between acquisitions of image frames;

generating multiple panorama images including joining image frames of the scene according to spatial relationships and determining stereoscopic counterpart relationships between the multiple panorama images;

processing the multiple panorama images based on the stereoscopic counterpart relationships to form a stereoscopic panorama image; and

storing, transmitting or displaying said stereoscopic panorama image, or combinations thereof.

12. The device of claim 11, wherein the determining of displacements comprises measuring the displacements.

13. The device of claim 11, wherein the determining of displacements comprise determining displacements in at least two dimensions.

14. The device of claim 13, wherein the at least two dimensions comprise two dimensions orthogonal to the depth dimension.

15. The device of claim 14, wherein the determining of displacements comprise determining displacements in three dimensions including the depth dimension.

16. The device of claim 11, wherein the determining stereoscopic counterpart relationships comprises:

pairing image frames having relative displacements within a first predetermined range configured to provide a selected stereoscopic effect;

combining paired image frames to form component stereoscopic images; and

retaining paired image frames having overlap with adjacent paired image frames within a second predetermined range.

17. The device of claim 16, wherein the first predetermined range comprises 5-7.5 cm.

18. The device of claim 11, wherein the partially overlapping comprises between 10% and 50% overlap.

19. The device of claim 11, wherein the method further comprises interleaving joining of pairs of image frames with acquiring or generating, or both, of next image frames.

20. The device of claim 11, wherein the first predetermined range is not greater than 15% of a panning radius of the imaging device.

21. One or more computer-readable storage media having code embedded therein for programming a processor to perform a method for generating a stereoscopic panorama image using a portable imaging device, said method comprising:

fixing an exposure level for acquiring the panorama image with the portable imaging device;

panning the imaging device across a scene;

acquiring multiple at least partially overlapping image frames of portions of said scene, including using an optic and imaging sensor of the portable imaging device;

registering the image frames, including determining displacements of the imaging device between acquisitions of image frames;

generating multiple panorama images including joining image frames of the scene according to spatial relationships and determining stereoscopic counterpart relationships between the multiple panorama images;

processing the multiple panorama images based on the stereoscopic counterpart relationships to form a stereoscopic panorama image; and

storing, transmitting or displaying said stereoscopic panorama image, or combinations thereof.

22. The one or more computer-readable storage media of claim 21, wherein the determining of displacements comprises measuring the displacements.

23. The one or more computer-readable storage media of claim 21, wherein the determining of displacements comprise determining displacements in at least two dimensions.

24. The one or more computer-readable storage media of claim 23, wherein the at least two dimensions comprise two dimensions orthogonal to the depth dimension.

25. The one or more computer-readable storage media of claim 24, wherein the determining of displacements comprise determining displacements in three dimensions including the depth dimension.

26. The one or more computer-readable storage media of claim 21, wherein the determining stereoscopic counterpart relationships comprises:

pairing image frames having relative displacements within a first predetermined range configured to provide a selected stereoscopic effect;

combining paired image frames to form component stereoscopic images; and

retaining paired image frames having overlap with adjacent paired image frames within a second predetermined range.

27. The one or more computer-readable storage media of claim 26, wherein the first predetermined range comprises 5-7.5 cm.

28. The one or more computer-readable storage media of claim 21, wherein the partially overlapping comprises between 10% and 50% overlap.

29. The one or more computer-readable storage media of claim 21, wherein the method further comprises interleaving joining of pairs of image frames with acquiring or generating, or both, of next image frames.

30. The one or more computer-readable storage media of claim 21, wherein the first predetermined range is not greater than 15% of a panning radius of the imaging device.

31. A method for generating a panorama image using a portable imaging device, said method comprising:

fixing an exposure level for acquiring the panorama image with the portable imaging device;

panning the portable imaging device across a scene;

acquiring at least two sets of images each including at least two image frames of portions of said scene and processing said sets, said acquiring comprising using an optic and imaging sensor of the portable imaging device, said processing including;

sorting and retaining the at least two image frames of the at least two sets;

determining relative displacements between substantially overlapping frames within the at least two sets of image frames;

registering images within the at least two sets relative to one another;

combining the substantially overlapping frames;

joining component images of the panorama image; and

storing, transmitting or displaying said panorama image, or combinations thereof.

32. The method of claim 31, further comprising determining a relative displacement between a first or otherwise corresponding acquired frame of each of said two or more sets of image frames.

33. The method of claim 31, further comprising registering a combined panoramic image derived from images of each of said two or more sets.

34. The method of claim 31, further comprising interleaving joining of pairs of image frames with acquiring or generating, or both, of next image frames.

35. A portable camera-enabled device capable of in-camera generation of a panorama image, comprising:

a lens;

an image sensor;

a processor; and

a processor readable medium having code embedded therein for programming the processor to perform a panorama image generation method that comprises;

fixing an exposure level for acquiring the panorama image with the portable imaging device;

panning the portable imaging device across a scene;

acquiring at least two sets of images each including at least two image frames of portions of said scene and processing said sets, said acquiring comprising using an optic and imaging sensor of the portable imaging device, said processing including;

sorting and retaining the at least two image frames of the at least two sets;

determining relative displacements between substantially overlapping frames within the at least two sets of image frames;

registering images within the at least two sets relative to one another;

combining the substantially overlapping frames;

joining component images of the panorama image; and

storing, transmitting or displaying said panorama image, or combinations thereof.

36. The device of claim 35, wherein the method further comprises determining a relative displacement between a first or otherwise corresponding acquired frame of each of said two or more sets of image frames.

37. The device of claim 35, wherein the method further comprises registering a combined panoramic image derived from images of each of said two or more sets.

38. The device of claim 35, wherein the method further comprises interleaving joining of pairs of image frames with acquiring or generating, or both, of next image frames.

39. One or more computer-readable storage media having code embedded therein for programming a processor to perform a method for generating a panorama image using a portable imaging device, said method comprising:

fixing an exposure level for acquiring the panorama image with the portable imaging device;

panning the portable imaging device across a scene;

acquiring at least two sets of images each including at least two image frames of portions of said scene and processing said sets, said acquiring comprising using an optic and imaging sensor of the portable imaging device, said processing including;

sorting and retaining the at least two image frames of the at least two sets;

determining relative displacements between substantially overlapping frames within the at least two sets of image frames;

registering images within the at least two sets relative to one another;

combining the substantially overlapping frames;

joining component images of the panorama image; and

storing, transmitting or displaying said panorama image, or combinations thereof.

40. The one or more computer-readable storage media of claim 39, wherein the method further comprises determining a relative displacement between a first or otherwise corresponding acquired frame of each of said two or more sets of image frames.

41. The one or more computer-readable storage media of claim 39, wherein the method further comprises registering a combined panoramic image derived from images of each of said two or more sets.

42. The one or more computer-readable storage media of claim 39, wherein the method further comprises interleaving joining of pairs of image frames with acquiring or generating, or both, of next image frames.