Motion adaptive image slice selection

• US 10,306,140 B2
• Filed: 06/06/2012
• Issued: 05/28/2019
• Est. Priority Date: 06/06/2012
• Status: Active Grant

• Current Assignee: Apple Inc.
• Original Assignee: Apple Inc.
• Inventors: Tico, Marius, Doepke, Frank
• Primary Examiner(s): Hossain, Farzana
• Application Number:
  US13/490,031
• Publication Number:
  US 20130329001A1
• Time in Patent Office: 2,547 Days
• Field of Search: 348 36
• CPC Class Codes: H04N 23/698 for achieving an enlarged field of view, e.g. panoramic image capture

Abstract

Systems, methods, and computer readable media for adaptively selecting what portion (aka slice) of a first image (aka frame) is selected to overlap and blend with a second frame during frame capture operations are disclosed. In general, for every new frame captured in a sequence the overlap between it and the slice selected from a prior frame may be determined based, at least in part, on sensor output. If the overlap so determined is below a desired threshold, the position of the current frame'"'"'s slice may be adjusted so as to provide the desired overlap.

Claims

1. A non-transitory program storage device, readable by a programmable processor and comprising instructions stored thereon to cause the processor to:

acquire a first frame and a second frame from an image capture device;

acquire, from one or more sensor devices, motion information corresponding to the image capture device'"'"'s motion during the acquisition of the first and second frames;

identify, based on the motion information, a first slice region in the first frame and a second slice region in the second frame, the second slice region'"'"'s center being substantially equal to the second frame'"'"'s center;

determine, based on the motion information and within an area of overlap between the first and second frames, whether the first and second slice regions overlap each other by at least a specified amount;

move, based on the motion information and in response to a determination that the first and second slice regions do not overlap each other by at least the specified amount, the second slice region in the second frame in a direction opposite to a direction of the image capture device'"'"'s motion information so that the second slice region'"'"'s center is spaced apart from the second frame'"'"'s center and so that the moved second slice region in the second frame overlaps with the first slice region in the first frame by at least the specified amount; and

perform a blend operation in the overlap between the first slice region and the moved second slice region to generate a single panoramic image.

2. The non-transitory program storage device of claim 1, the first and second slice regions have a same width.

3. The non-transitory program storage device of claim 2, wherein the specified amount is between 20% and 90% of the width.

4. The non-transitory program storage device of claim 1, wherein the one or more sensor devices are proximate to or affixed to the image capture device.

5. The non-transitory program storage device of claim 1, wherein the instructions to cause the processor to acquire motion information comprise instructions to cause the processor to acquire the motion information from metadata from at least one of the first and second frames.

6. The non-transitory program storage device of claim 2, wherein the width is determined based on one or more of:

a speed of acquiring the first and second frames;

an amount of available computational resources; and

an image quality of a lens of the image capture device.

7. The non-transitory program storage device of claim 1, wherein the first and second frames are not acquired successively.

8. The non-transitory program storage device of claim 1, wherein the instructions to cause the processor to perform the blend operation comprise instructions to cause the processor to perform the blend operation of the first and second frames immediately after moving the second slice region so as to have the overlap by at least the specified amount.

9. The non-transitory program storage device of claim 8, wherein the instructions to cause the processor to perform the blend operation further comprise instructions to cause the processor to display the blended frames.

10. The non-transitory program storage device of claim 9, wherein the instructions to cause the processor to display the blended frames comprise instructions to cause the processor to display the blended frames on a display of the image capture device.

11. The non-transitory program storage device of claim 1, wherein the first slice region'"'"'s center is substantially equal to the first frame'"'"'s center.

12. A method to generate a wide angle-of-view image, comprising:

acquiring a first frame and a second frame from an image capture device;

acquiring, from one or more sensor devices, motion information corresponding to the image capture device'"'"'s motion during the acquisition of the first and second frames;

identifying, based on the motion information, a first slice region in the first frame and a second slice region in the second frame, the second slice region'"'"'s center being substantially equal to the second frame'"'"'s center;

determining, based on the motion information and within an area of overlap between the first and second frames, whether the first and second slice regions overlap each other by at least a specified amount;

moving, based on the motion information and in response to a determination that the first and second slice regions do not overlap each other by at least the specified amount, the second slice region in the second frame in a direction opposite to a direction of the image capture device'"'"'s motion information so that the second slice region'"'"'s center is spaced apart from the second frame'"'"'s center and so that the moved second slice region in the second frame overlaps with the first slice region in the first frame by at least the specified amount; and

performing a blend operation in the overlap between the first slice region and the moved second slice region to generate a single panoramic image.

13. The method of claim 12, wherein the first and second slice regions have a same width.

14. The method of claim 13, wherein the specified amount is between 20% and 90% of the width.

15. The method of claim 12, wherein the one or more sensor devices are proximate to or affixed to the image capture device.

16. The method of claim 12, wherein acquiring motion information corresponding to the image capture device'"'"'s motion comprises acquiring motion information from metadata from at least one of the first and second frames.

17. The method of claim 16, wherein the metadata comprises metadata captured by the image capture device concomitant with capture of at least one of the first and second frames.

18. The method of claim 13, wherein the width is determined based on one or more of:

a speed of acquiring the first and second frames;

an amount of available computational resources; and

an image quality of a lens of the image capture device.

19. The method of claim 12, wherein the first and second frames are not acquired successively.

20. The method of claim 12, wherein performing the blend operation comprises blending the first and second frames immediately after moving the second slice region so as to have the overlap by at least the specified amount.

21. The method of claim 20, wherein performing the blend operation further comprises displaying the blended frames.

22. The method of claim 21, wherein displaying the blended frames comprises displaying the blended frames on a display element of the image capture device.

23. The method of claim 12, wherein the first slice region'"'"'s center is substantially equal to the first frame'"'"'s center.

24. An electronic device comprising:

an image capture unit;

one or more sensor devices;

memory operatively coupled to the image capture unit;

a processor operatively coupled to the image capture unit, the one or more sensor devices and the memory, wherein the memory stores data, wherein the data comprises instructions, and wherein the processor is configured to execute the instructions to;

acquire a first frame and a second frame from the image capture unit;

acquire, from the one or more sensor devices, motion information corresponding to the image capture unit'"'"'s motion during the acquisition of the first and second frames;

identify, based on the motion information, a first slice region in the first frame and a second slice region in the second frame, the second slice region'"'"'s center being substantially equal to the second frame'"'"'s center;

determine, based on the motion information and within an area of overlap between the first and second frames, whether the first and second slice regions overlap each other by at least a specified amount;

move, based on the motion information and in response to a determination that the first and second slice regions do not overlap each other by at least the specified amount, the second slice region in the second frame in a direction opposite to a direction of the image capture unit'"'"'s motion information so that the second slice region'"'"'s center is spaced apart from the second frame'"'"'s center and so that the moved second slice region in the second frame overlaps with the first slice region in the first frame by at least the specified amount; and

perform a blend operation in the overlap between the first slice region and the moved second slice region to generate a single panoramic image.

25. The device of claim 24, further comprising a display element, wherein the memory further has instructions stored therein to cause the processor to:

blend the first and second frames immediately after moving the second slice region so as to have the overlap by at least the specified amount; and

display, on the display element, the blended frames.

26. The device of claim 24, wherein the first slice region'"'"'s center is substantially equal to the first frame'"'"'s center.

27. The device of claim 24, wherein the first and second slice regions have a same width.