Constructing image panorama using frame selection

• US 7,860,343 B2
• Filed: 04/10/2006
• Issued: 12/28/2010
• Est. Priority Date: 04/10/2006
• Status: Active Grant

• Current Assignee: Nokia Technologies Oy (Nokia Corporation)
• Original Assignee: Nokia Corporation
• Inventors: Boutellier, Jani, Vehviläinen, Markku, Silven, Olli, Tico, Marius
• Primary Examiner(s): Perungavoor; Sath V
• Assistant Examiner(s): Cunningham; Gregory F
• Application Number:
  US11/402,306
• Publication Number:
  US 20070237423A1
• Time in Patent Office: 1,723 Days
• Field of Search: 348/36, 382/284
• US Class Current: 382/284
• CPC Class Codes: G06T 2207/30168 Image quality inspection
• CPC Class Codes: G06T 3/4038 Image mosaicing, e.g. composing plane images from plane sub-images
• CPC Class Codes: G06T 7/0002 Inspection of images, e.g. flaw detection
• CPC Class Codes: H04N 2101/00 Still video cameras
• CPC Class Codes: H04N 23/661 Transmitting camera control signals through networks, e.g. control via the Internet
• CPC Class Codes: H04N 23/698 for achieving an enlarged field of view, e.g. panoramic image capture
• CPC Class Codes: H04N 23/951 by using two or more images to influence resolution, frame rate or aspect ratio

Abstract

A new method, system, apparatus and software product for constructing by an electronic device an image panorama from a plurality of images based on their visual quality is presented. The same portion of a panoramic image scene is typically captured on multiple neighborhood video frames from which one can choose the best visual representation to be pasted into the panorama. Constructing the image panorama from the plurality of captured images can comprise the following steps: a) evaluating image quality of K consecutive frames out of the captured plurality of images using a predetermined criterion, b) selecting one image out of the evaluated K frames with the best image quality using a predetermined rule, and c) appending the selected frame to previously selected frames for constructing the image panorama. On-line and off-line implementations are possible.

Claims

1. A method, comprising:

dividing a plurality of captured images into groups, each group comprising a number of consecutive frames;

evaluating by an electronic device, image quality of said number of consecutive frames for said each group out of the plurality of said captured images using a predetermined criterion, wherein said number is an integer of at least a value of two;

selecting by said electronic device, one frame out of said evaluated frames with the best image quality from said each group using a predetermined rule; and

appending by said electronic device, said one frame from said each group to other selected frames for constructing an image panorama.

2. The method of claim 1, wherein said number is a predetermined number of a constant value for all said groups.

3. The method of claim 1, wherein said number is variable for said groups.

4. The method of claim 3, wherein said number is selected in response to an amount of motion of a sensor capturing said K consecutive frames during said capturing using a pre-selected criterion.

5. The method of claim 3, wherein said number is assigned an initial value and then is iterated during said evaluating by monitoring a relative motion speed of features in adjacent frames using a pre-selected rule.

6. The method of claim 5, wherein, using said pre-selected rule, said number is increased by a pre-selected value when said relative motion speed is smaller than a threshold value or said number is decreased by a further pre-selected value when said relative motion speed is larger than a further threshold value.

7. The method of claim 1, wherein said plurality of images are stored in a memory to perform said evaluating, selecting and appending.

8. The method of claim 1, wherein said evaluating, selecting and appending are performed by the electronic device by accessing said plurality of images through network communications.

9. The method of claim 1, wherein during said evaluating image quality of said number of the consecutive frames only one frame at a time is stored in a memory.

10. The method of claim 1, wherein said evaluating image quality for each of said number of the consecutive frames uses the predetermined criterion by evaluating at least one of the following four image characteristics:

a sharpness,

a registration accuracy,

an amount of overlap with other frames,

a type and an amount of motion.

11. The method of claim 10, wherein said predetermined criterion comprises all four image characteristics.

12. The method of claim 1, wherein said evaluating, selecting and appending are performed by the electronic device, which is a digital camera, a communication device, a wireless communication device, a portable electronic device, a mobile electronic device or a camera phone.

13. The method of claim 1, wherein boundary of the selected frame are excluded and substantially only a central part of said selected frame is appended to said other selected frames.

14. The method of claim 1, wherein said selected frame is stored after said selecting and appended to said image panorama only after all frames to be appended to said image panorama out of said plurality of images are selected.

15. The method of claim 1, wherein said one frame is appended to previously selected frames right after said selecting.

16. A computer readable storage medium stored with code thereon for execution by a processor, wherein said code comprises instructions for performing the method of claim 1.

17. An electronic apparatus, comprising:

a processor and memory including computer program code, the memory and the computer program code configured to, with the processor, cause the electric apparatus at least to;

evaluate image quality for each group comprising a number of consecutive frames out of a plurality of images using a predetermined criterion, wherein said number is an integer of at least a value of two, wherein said plurality of captured images is divided into said groups;

select one frame out of said each group with the best image quality using a predetermined rule; and

append said one frame for each said group to other selected frames for constructing an image panorama.

18. The apparatus of claim 17, wherein the memory and the computer program code are further configured to, with the processor, cause the electronic apparatus at least to further perform:

store a selected frame after selection and

append said selected frame to said image panorama only after all frames to be appended to said image panorama out of said plurality of images are selected.

19. The apparatus of claim 17, wherein the memory and the computer program code are further configured to, with the processor, cause the electronic apparatus at least to further perform:

append said one frame to previously selected frames right after selection.

20. The apparatus of claim 17, wherein said number is a predetermined number of a constant value for all said groups.

21. The apparatus of claim 17, wherein said number is variable for said groups.

22. The apparatus of claim 21, wherein said number is selected in response to an amount of motion of a sensor capturing said K consecutive frames during said capturing using a pre-selected criterion.

23. The apparatus of claim 21, wherein said number is assigned an initial value and then is iterated during said evaluating by monitoring a relative motion speed of features in adjacent frames using a pre-selected rule.

24. The electronic apparatus of claim 17, wherein

the memory stores said plurality of images.

25. The apparatus of claim 17, wherein said apparatus accesses said plurality of images through network communications.

26. The apparatus of claim 25, wherein during evaluation of image quality of said number of the consecutive frames only one frame at a time is stored in said memory of said apparatus.

27. The apparatus of claim 17, wherein evaluation of image quality for each of said number of the consecutive frames uses the predetermined criterion by evaluating at least one of the following four image characteristics:

a sharpness,

a registration accuracy,

an amount of overlap with other frames,

a type and an amount of motion.

28. The electronic apparatus of claim 17, wherein said electronic apparatus is a digital camera, a communication device, a wireless communication device, a portable apparatus, a mobile electronic device or a camera phone.

29. A system adopted for constructing an image panorama from a plurality of images, comprising:

a camera that captures a plurality of images; and

an electronic device, comprising;

a frame image quality evaluation module that evaluates image quality for each group comprising a number of consecutive frames out of a plurality of images using a predetermined criterion, wherein each of said number of the consecutive frames is provided to the electronic device through a network communication said number being an integer of at least a value of two, and wherein said plurality of captured images is divided into said groups;

a frame selection module that selects one frame out of said each group with the best image quality using a predetermined rule; and

an image panorama construction block that appends said one frame for each said group to other selected frames for constructing an image panorama.

30. The system of claim 29, wherein said network communications are network communications over Internet.

31. The system of claim 29, wherein said means for capturing is a part of said electronic device or is separate from said electronic device.

32. An electronic apparatus, comprising:

means for evaluating image quality for each group comprising a number of consecutive frames out of a plurality of images using a predetermined criterion, wherein said number is an integer of at least a value of two, wherein said plurality of captured images is divided into said groups;

means for selecting one frame out of said each group with the best image quality using a predetermined rule; and

means for appending said one frame for each said group to other selected frames for constructing an image panorama.

33. The apparatus of claim 32, wherein said means for evaluating is a frame image quality evaluation module, said means for selecting is a frame selection module, and said means for appending is an image panorama construction block.