Constructing image panorama using frame selection

• US 20070237423A1
• Filed: 04/10/2006
• Published: 10/11/2007
• Est. Priority Date: 04/10/2006
• Status: Active Grant

• Current Assignee: Nokia Technologies Oy (Nokia Corporation)
• Original Assignee: Nokia Corporation
• Inventors: Tico, Marius, Vehvilainen, Markku, Boutellier, Jani, Silven, Olli
• Granted Patent: US 7,860,343 B2
• Time in Patent Office: Days
• US Class Current: 382/284
• CPC Class Codes: G06T 2207/30168 Image quality inspection
• CPC Class Codes: G06T 3/4038 Image mosaicing, e.g. composing plane images from plane sub-images
• CPC Class Codes: G06T 7/0002 Inspection of images, e.g. flaw detection
• CPC Class Codes: H04N 2101/00 Still video cameras
• CPC Class Codes: H04N 23/661 Transmitting camera control signals through networks, e.g. control via the Internet
• CPC Class Codes: H04N 23/698 for achieving an enlarged field of view, e.g. panoramic image capture
• CPC Class Codes: H04N 23/951 by using two or more images to influence resolution, frame rate or aspect ratio

Abstract

The specification and drawings present a new method, system, apparatus and software product for constructing by an electronic device an image panorama from a plurality of images based on their visual quality. The same portion of a panoramic image scene is typically captured on multiple neighborhood video frames from which one can choose the best visual representation to be pasted into the panorama. Constructing the image panorama from the plurality of captured images can comprise the following steps: a) evaluating image quality of K consecutive frames out of the captured plurality of images using a predetermined criterion, b) selecting one image out of the evaluated K frames with the best image quality using a predetermined rule, and c) appending the selected frame to previously selected frames for constructing the image panorama. On-line and off-line implementations are possible.

Claims

1. A method, comprising:

evaluating image quality of a number of consecutive frames out of a plurality of images using a predetermined criterion, wherein said number is an integer of at least a value of two;

selecting one frame out of said evaluated frames with the best image quality using a predetermined rule; and

appending said one frame to other selected frames for constructing an image panorama by an electronic device.

2. The method of claim 1, wherein said number is a predetermined number of a constant value.

3. The method of claim 1, wherein said number is variable.

4. The method of claim 3, wherein said number is selected in response to an amount of motion of a sensor capturing said K consecutive frames during said capturing using a pre-selected criterion.

5. The method of claim 3, wherein said number is assigned an initial value and then is iterated during said evaluating by monitoring a relative motion speed of features in adjacent frames using a pre-selected rule.

6. The method of claim 5, wherein, using said pre-selected rule, said number is increased by a pre-selected value when said relative motion speed is smaller than a threshold value or said number is decreased by a further pre-selected value when said relative motion speed is larger than a further threshold value.

7. The method of claim 1, wherein said electronic device comprises a memory for storing said plurality of images to perform said steps of evaluating, selecting and appending.

8. The method of claim 1, wherein said electronic device is configured to access said plurality of images through network communications.

9. The method of claim 1, wherein during said evaluating image quality of said number of the consecutive frames only one frame at a time is saved in a memory of said electronic device.

10. The method of claim 1, wherein said evaluating image quality for each of said number of the consecutive frames uses the predetermined criterion by evaluating at least one of the following four image characteristics:

a) a sharpness,

b) a registration accuracy,

c) an amount of overlap with other frames,

d) a type and an amount of motion.

11. The method of claim 10, wherein said predetermined criterion comprises all four image characteristics.

12. The method of claim 1, wherein said electronic device is a digital camera, a communication device, a wireless communication device, a portable electronic device, a mobile electronic device or a camera phone.

13. The method of claim 1, wherein boundary of the selected frame are excluded and substantially only a central part of said selected frame is appended to said other selected frames.

14. The method of claim 1, wherein said selected frame is stored after said selecting and appended to said image panorama only after all frames to be appended to said image panorama out of said plurality of images are selected.

15. The method of claim 1, wherein said one frame is appended to previously selected frames right after said selecting.

16. A computer program product comprising:

a computer readable storage structure embodying computer program code thereon for execution by a computer processor with said computer program code, wherein said computer program code comprises instructions for performing the method of claim 1, indicated as being performed by any component of the electronic device.

17. An electronic device, comprising:

a frame image quality evaluation module, configured to evaluate image quality of a number of consecutive frames out of a plurality of images using a predetermined criterion, wherein said number is an integer of at least a value of two;

a frame selection module, configured to select one frame out of said evaluated frames with the best image quality using a predetermined rule; and

an image panorama construction block, configured to append said one frame to other selected frames for constructing an image panorama.

18. The electronic device of claim 17, wherein said electronic device is configured to store selected frame after said selecting and to append said selected frame to said image panorama only after all frames to be appended to said image panorama out of said plurality of images are selected.

19. The electronic device of claim 17, wherein said electronic device is configured to append said one frame to previously selected frames right after said selecting.

20. The electronic device of claim 17, wherein said number is a predetermined number of a constant value.

21. The electronic device of claim 17, wherein said number is variable.

22. The electronic device of claim 21, wherein said number is selected in response to an amount of motion of a sensor capturing said K consecutive frames during said capturing using a pre-selected criterion.

23. The electronic device of claim 21, wherein said number is assigned an initial value and then is iterated during said evaluating by monitoring a relative motion speed of features in adjacent frames using a pre-selected rule.

24. The electronic device of claim 17, further comprising:

a memory for storing said plurality of images for performing said evaluating, selecting and appending.

25. The electronic device of claim 17, wherein said electronic device is configured to access said plurality of images through network communications.

26. The electronic device of claim 25, wherein during said evaluating image quality of said number of the consecutive frames only one frame at a time is saved in a memory of said electronic device.

27. The electronic device of claim 17, wherein said evaluating image quality for each of said number of the consecutive frames uses the predetermined criterion by evaluating at least one of the following four image characteristics:

e) a sharpness,

f) a registration accuracy,

g) an amount of overlap with other frames,

h) a type and an amount of motion.

28. The method of claim 17, wherein said electronic device is a digital camera, a communication device, a wireless communication device, a portable electronic device, a mobile electronic device or a camera phone.

29. A system adopted for constructing an image panorama from a plurality of images, comprising:

means for capturing a plurality of images; and

an electronic device, comprising;

a frame image quality evaluation module, configured to evaluate image quality of a number of consecutive frames out of said plurality of images using a predetermined criterion, wherein each of said number of the consecutive frames is provided to the electronic device through a network communication, and said number is an integer of at least a value of two;

a frame selection module, configured to select one frame out of said evaluated frames with the best image quality using a predetermined rule; and

an image panorama construction block, configured to append said one frame to other selected frames for constructing an image panorama.

30. The system of claim 29, wherein said network communications are network communications over Internet.

31. The system of claim 29, wherein said means for capturing is a part of said electronic device or is separate from said electronic device.

32. An electronic device, comprising:

means for evaluating image quality of a number of consecutive frames out of a plurality of images using a predetermined criterion, wherein said number is an integer of at least a value of two;

means for selecting one frame out of said evaluated frames with the best image quality using a predetermined rule; and

means for appending said one frame to other selected frames for constructing an image panorama.

33. The electronic device of claim 32, wherein said means for evaluating is a frame image quality evaluation module, said means for selecting is a frame selection module, and said means for appending is an image panorama construction block.