Image processing device, image processing method and storage medium

• US 8,538,192 B2
• Filed: 10/26/2010
• Issued: 09/17/2013
• Est. Priority Date: 10/26/2010
• Status: Active Grant

• Current Assignee: Morpho Incorporated
• Original Assignee: Morpho Incorporated
• Inventors: Habuka, Kensuke, Shino, Toshiki
• Primary Examiner(s): COUSO, YON JUNG
• Application Number:
  US13/260,278
• Publication Number:
  US 20120177302A1
• Time in Patent Office: 1,057 Days
• Field of Search: 382/274, 382/282, 382/284, 382/294, 382/190, 382/195, 382/199, 382/256, 382/266
• US Class Current: 382/274
• CPC Class Codes: G06T 3/4038 Image mosaicing, e.g. composing plane images from plane sub-images
• CPC Class Codes: H04N 1/3876 Recombination of partial images to recreate the original image
• CPC Class Codes: H04N 5/2624 for obtaining an image which is composed of whole input images, e.g. splitscreen

Abstract

An image processing device that sequentially generates a synthetic image by joining a first image, which is a single image or constituted by a plurality of the images joined together, and a second image that is input, every time when the second image is input, includes: an overlapping region obtainment portion 14 that obtains a motion vector on the basis of the second image and an image input immediately before the second image among the images constituting the first image, and obtains an overlapping region where the image input immediately before and the second image overlap each other on the basis of the obtained motion vector; a brightness-value conversion factor calculation portion 16 that calculates a brightness conversion factor for converting a brightness value of the first image or a brightness value of the second image on the basis of the brightness value of the first image and the brightness value of the second image in the overlapping region so that a change in the brightness values between the first image and the second image becomes small; and a synthetic image generation portion 12 that generates the synthetic image by converting the first image or the second image using the brightness conversion factor and by joining the first image and the second image.

Claims

1. An image processing device that sequentially generates a synthetic image by joining a first image, which is a single image or is constituted by a plurality of the images joined together, and a second image that is input, every time when the second image is input, the device comprising:

an overlapping region obtainment portion that obtains a motion vector on the basis of the second image and an image input immediately before the second image among the images constituting the first image, and obtains an overlapping region where the image input immediately before and the second image overlap each other on the basis of the obtained motion vector;

a brightness-value conversion factor calculation portion that calculates a brightness conversion factor for converting a brightness value of the first image or a brightness value of the second image on the basis of the brightness value of the first image and the brightness value of the second image in the overlapping region so that a change in the brightness values between the first image and the second image becomes small; and

a synthetic image generation portion that generates the synthetic image by converting the first image or the second image using the brightness conversion factor and by joining the first image and the second image,

wherein the synthetic image generation portion determines the brightness-value conversion factor at the synthetic position on the basis of the distance from the joint at the synthetic position, derives a correction function that converts the brightness value of the second image on the basis of the determined brightness-value conversion factor and calculates the brightness value of the second image used at the synthetic position on the basis of the correction function and the brightness value of the second image.

2. The image processing device according to claim 1, further comprising:

a brightness absolute-value calculation portion that calculates a brightness absolute value of the second image, the brightness absolute value being a ratio of the total sum of the brightness values of a predetermined reference image in the first image to the total sum of the brightness values of the second image on the basis of the total sum of the brightness values of the image input immediately before and the total sum of the brightness values of the second image in the overlapping region, wherein

the brightness-value conversion factor calculation portion calculates, on the basis of the brightness absolute value of a target image overlapping the second image among the images constituting the first image and the brightness absolute value of the second image, the brightness value conversion factor of the second image that depends on a distance from a joint between the target image and the second image.

3. The image processing device according to claim 1, wherein

the brightness-value conversion factor calculation portion calculates the brightness-value conversion factor so that a weight of the second image at a synthetic position becomes large in proportion to the distance from the joint.

4. The image processing device according to claim 3, wherein

the correction function is a linear function with the brightness-value conversion factor as a proportional constant in a range in which a pixel value before the conversion and the pixel value after the conversion are less than a predetermined threshold value and a linear function continued from the linear function up to the threshold value and passing through the maximum pixel value in a range in which the pixel value before the conversion or the pixel value after the conversion is equal to more than the predetermined threshold value.

5. The image processing device according to claim 1, wherein

the correction function is a linear function with the brightness-value conversion factor as a proportional constant in a range in which a pixel value before the conversion and the pixel value after the conversion are less than a predetermined threshold value and a linear function continued from the linear function up to the threshold value and passing through the maximum pixel value in a range in which the pixel value before the conversion or the pixel value after the conversion is equal to more than the predetermined threshold value.

6. The image processing device according to claim 5, further comprising:

a center-position obtainment portion that obtains the motion vector on the basis of the image input immediately before the second image among the images constituting the first image and the second image, and obtains position information of a first center point, which is a center point of each of the images constituting the first image, and position information of a second center point, which is a center point of the second image, on the basis of the obtained motion vector, wherein

the synthetic image generation portion obtains the first center point of the image overlapping the second image among the images constituting the first image and generates the synthetic image by joining the first image and the second image using a perpendicular bisector between the obtained first center point and the second center point as a joint between the first image and the second image on the basis of the obtained position information of the first center point and the position information of the second center point.

7. The image processing device according to claim 1, further comprising:

a center-position obtainment portion that obtains the motion vector on the basis of the image input immediately before the second image among the images constituting the first image and the second image, and obtains position information of a first center point, which is a center point of each of the images constituting the first image, and position information of a second center point, which is a center point of the second image, on the basis of the obtained motion vector, wherein

the synthetic image generation portion obtains the first center point of the image overlapping the second image among the images constituting the first image and generates the synthetic image by joining the first image and the second image using a perpendicular bisector between the obtained first center point and the second center point as a joint between the first image and the second image on the basis of the obtained position information of the first center point and the position information of the second center point.

8. The image processing device according to claim 7, wherein

the synthetic image generation portion makes a pixel value within a range from the joint between the first image and the second image to a position separated from the joint by a predetermined distance and within a range smaller than the overlapping region, a synthetic value of the pixel value of the first image and the pixel value of the second image.

9. An image processing device that sequentially generates a synthetic image by joining a first image, which is a single image or is constituted by a plurality of the images joined together, and a second image that is input, every time when the second image is input, the device comprising:

an overlapping region obtainment portion that obtains a motion vector on the basis of the second image and an image input immediately before the second image among the images constituting the first image, and obtains an overlapping region where the image input immediately before and the second image overlap each other on the basis of the obtained motion vector;

a brightness-value conversion factor calculation portion that calculates a brightness conversion factor for converting a brightness value of the first image or a brightness value of the second image on the basis of the brightness value of the first image and the brightness value of the second image in the overlapping region so that a change in the brightness values between the first image and the second image becomes small; and

a synthetic image generation portion that generates the synthetic image by converting the first image or the second image using the brightness conversion factor and by joining the first image and the second image,

wherein the brightness-value conversion factor calculation portion calculates the brightness-value conversion factor so that a weight of the second image at a synthetic position becomes large in proportion to the distance from the joint.

10. The image processing device according to claim 9, further comprising:

a brightness absolute-value calculation portion that calculates a brightness absolute value of the second image, the brightness absolute value being a ratio of the total sum of the brightness values of a predetermined reference image in the first image to the total sum of the brightness values of the second image on the basis of the total sum of the brightness values of the image input immediately before and the total sum of the brightness values of the second image in the overlapping region, wherein

the brightness-value conversion factor calculation portion calculates, on the basis of the brightness absolute value of a target image overlapping the second image among the images constituting the first image and the brightness absolute value of the second image, the brightness value conversion factor of the second image that depends on a distance from a joint between the target image and the second image.

11. The image processing device according to any one of claim 9, further comprising:

a center-position obtainment portion that obtains the motion vector on the basis of the image input immediately before the second image among the images constituting the first image and the second image, and obtains position information of a first center point, which is a center point of each of the images constituting the first image, and position information of a second center point, which is a center point of the second image, on the basis of the obtained motion vector, wherein

the synthetic image generation portion obtains the first center point of the image overlapping the second image among the images constituting the first image and generates the synthetic image by joining the first image and the second image using a perpendicular bisector between the obtained first center point and the second center point as a joint between the first image and the second image on the basis of the obtained position information of the first center point and the position information of the second center point.

12. An image processing device that sequentially generates a synthetic image by joining a first image, which is a single image or is constituted by a plurality of the images joined together, and a second image that is input, every time when the second image is input, the device comprising:

an overlapping region obtainment portion that obtains a motion vector on the basis of the second image and an image input immediately before the second image among the images constituting the first image, and obtains an overlapping region where the image input immediately before and the second image overlap each other on the basis of the obtained motion vector;

a brightness-value conversion factor calculation portion that calculates a brightness conversion factor for converting a brightness value of the first image or a brightness value of the second image on the basis of the brightness value of the first image and the brightness value of the second image in the overlapping region so that a change in the brightness values between the first image and the second image becomes small;

a center-position obtainment portion that obtains the motion vector on the basis of the image input immediately before the second image among the images constituting the first image and the second image, and obtains position information of a first center point, which is a center point of each of the images constituting the first image, and position information of a second center point, which is a center point of the second image, on the basis of the obtained motion vector; and

a synthetic image generation portion that generates the synthetic image by converting the first image or the second image using the brightness conversion factor and by joining the first image and the second image,

wherein the synthetic image generation portion obtains the first center point of the image overlapping the second image among the images constituting the first image and generates the synthetic image by joining the first image and the second image using a perpendicular bisector between the obtained first center point and the second center point as a joint between the first image and the second image on the basis of the obtained position information of the first center point and the position information of the second center point.

13. The image processing device according to claim 12, further comprising:

a brightness absolute-value calculation portion that calculates a brightness absolute value of the second image, the brightness absolute value being a ratio of the total sum of the brightness values of a predetermined reference image in the first image to the total sum of the brightness values of the second image on the basis of the total sum of the brightness values of the image input immediately before and the total sum of the brightness values of the second image in the overlapping region, wherein

the brightness-value conversion factor calculation portion calculates, on the basis of the brightness absolute value of a target image overlapping the second image among the images constituting the first image and the brightness absolute value of the second image, the brightness value conversion factor of the second image that depends on a distance from a joint between the target image and the second image.

14. The image processing device according to claim 12, wherein

the synthetic image generation portion makes a pixel value within a range from the joint between the first image and the second image to a position separated from the joint by a predetermined distance and within a range smaller than the overlapping region, a synthetic value of the pixel value of the first image and the pixel value of the second image.

15. An image processing method that sequentially generates a synthetic image by joining a first image, which is a single image or is constituted by a plurality of the images joined together, and a second image that is input, every time when the second image is input, the method comprising causing a computer to perform the steps of:

obtaining a motion vector on the basis of the second image and an image input immediately before the second image among the images constituting the first image, and obtaining an overlapping region where the image input immediately before and the second image overlap each other on the basis of the obtained motion vector;

calculating, on the basis of a brightness value of the first image and a brightness value of the second image, a brightness value conversion factor for converting the brightness value of the first image or the brightness value of the second image on the basis of the brightness value of the first image and the brightness value of the second image in the overlapping region so that a change in the brightness values between the first image and the second image becomes small; and

generating the synthetic image by converting the first image or the second image using the brightness conversion factor and by joining the first image and the second image,

wherein the step of generating the synthetic image comprising the steps of;

determining the brightness-value conversion factor at the synthetic position on the basis of the distance from the joint at the synthetic position;

deriving a correction function that converts the brightness value of the second image on the basis of the determined brightness-value conversion factor; and

calculating the brightness value of the second image used at the synthetic position on the basis of the correction function and the brightness value of second image.

16. An image processing method that sequentially generates a synthetic image by joining a first image, which is a single image or is constituted by a plurality of the images joined together, and a second image that is input, every time when the second image is input, the method comprising causing a computer to perform the steps of:

obtaining a motion vector on the basis of the second image and an image input immediately before the second image among the images constituting the first image, and obtaining an overlapping region where the image input immediately before and the second image overlap each other on the basis of the obtained motion vector;

calculating, on the basis of a brightness value of the first image and a brightness value of the second image, a brightness value conversion factor for converting the brightness value of the first image or the brightness value of the second image on the basis of the brightness value of the first image and the brightness value of the second image in the overlapping region so that a change in the brightness values between the first image and the second image becomes small; and

generating the synthetic image by converting the first image or the second image using the brightness conversion factor and by joining the first image and the second image,

wherein the step of calculating comprising the steps of calculating the brightness-value conversion factor calculation portion calculates the brightness-value conversion factor so that a weight of the second image at a synthetic position becomes large in proportion to the distance from the joint.

17. An image processing program embodied on a computer readable medium that causes a computer to function so as to sequentially generate a synthetic image by joining a first image, which is a single image or is constituted by a plurality of the images joined together, and a second image that is input, every time when the second image is input, the program causing a computer to function as:

an overlapping region obtainment portion that obtains a motion vector on the basis of the second image and an image input immediately before the second image among the images constituting the first image, and obtains an overlapping region where the image input immediately before and the second image overlap each other on the basis of the obtained motion vector;

a brightness-value conversion factor calculation portion that calculates a brightness conversion factor for converting a brightness value of the first image or a brightness value of the second image on the basis of the brightness value of the first image and the brightness value of the second image in the overlapping region so that a change in the brightness values between the first image and the second image becomes small; and

a synthetic image generation portion that generates the synthetic image by converting the first image or the second image using the brightness conversion factor and by joining the first image and the second image,

wherein the synthetic image generation portion determines the brightness-value conversion factor at the synthetic position on the basis of the distance from the joint at the synthetic position, derives a correction function that converts the brightness value of the second image on the basis of the determined brightness-value conversion factor and calculates the brightness value of the second image used at the synthetic position on the basis of the correction function and the brightness value of the second image.

18. An image processing method that sequentially generates a synthetic image by joining a first image, which is a single image or is constituted by a plurality of the images joined together, and a second image that is input, every time when the second image is input, the method comprising the steps of:

obtaining a motion vector on the basis of the second image and an image input immediately before the second image among the images constituting the first image, and obtaining an overlapping region where the image input immediately before and the second image overlap each other on the basis of the obtained motion vector;

calculating, on the basis of a brightness value of the first image and a brightness value of the second image, a brightness value conversion factor for converting the brightness value of the first image or the brightness value of the second image on the basis of the brightness value of the first image and the brightness value of the second image in the overlapping region so that a change in the brightness values between the first image and the second image becomes small;

obtaining the motion vector on the basis of the image input immediately before the second image among the images constituting the first image and the second image;

obtaining position information, of a first center point, which is a center point of each of the images constituting the first image, and position information of a second center point, which is a center point of the second image, on the basis of the obtained motion vector; and

generating the synthetic image by converting the first image or the second image using the brightness conversion factor and by joining the first image and the second image,

wherein the step of generating the synthetic image comprising the steps of;

obtaining the first center point of the image overlapping the second image among the images constituting the first image; and

generating the synthetic image by joining the first image and the second image using a perpendicular bisector between the obtained first center point and the second center point as a joint between the first image and the second image on the basis of the obtained position information of the first center point and the position information of the second center point.

19. An image processing program that causes a computer to function so as to sequentially generate a synthetic image by joining a first image, which is a single image or is constituted by a plurality of the images joined together, and a second image that is input, every time when the second image is input, the program causing a computer to function as:

an overlapping region obtainment portion that obtains a motion vector on the basis of the second image and an image input immediately before the second image among the images constituting the first image, and obtains an overlapping region where the image input immediately before and the second image overlap each other on the basis of the obtained motion vector;

a brightness-value conversion factor calculation portion that calculates a brightness conversion factor for converting a brightness value of the first image or a brightness value of the second image on the basis of the brightness value of the first image and the brightness value of the second image in the overlapping region so that a change in the brightness values between the first image and the second image becomes small; and

a synthetic image generation portion that generates the synthetic image by converting the first image or the second image using the brightness conversion factor and by joining the first image and the second image,

the brightness-value conversion factor calculation portion calculates the brightness-value conversion factor so that a weight of the second image at a synthetic position becomes large in proportion to the distance from the joint.

20. An image processing program that causes a computer to function so as to sequentially generate a synthetic image by joining a first image, which is a single image or is constituted by a plurality of the images joined together, and a second image that is input, every time when the second image is input, the program causing a computer to function as:

an overlapping region obtainment portion that obtains a motion vector on the basis of the second image and an image input immediately before the second image among the images constituting the first image, and obtains an overlapping region where the image input immediately before and the second image overlap each other on the basis of the obtained motion vector;

a brightness-value conversion factor calculation portion that calculates a brightness conversion factor for converting a brightness value of the first image or a brightness value of the second image on the basis of the brightness value of the first image and the brightness value of the second image in the overlapping region so that a change in the brightness values between the first image and the second image becomes small;

a center-position obtainment portion that obtains the motion vector on the basis of the image input immediately before the second image among the images constituting the first image and the second image, and obtains position information of a first center point, which is a center point of each of the images constituting the first image, and position information of a second center point, which is a center point of the second image, on the basis of the obtained motion vector; and

a synthetic image generation portion that generates the synthetic image by converting the first image or the second image using the brightness conversion factor and by joining the first image and the second image,

the synthetic image generation portion obtains the first center point of the image overlapping the second image among the images constituting the first image and generates the synthetic image by joining the first image and the second image using a perpendicular bisector between the obtained first center point and the second center point as a joint between the first image and the second image on the basis of the obtained position information of the first center point and the position information of the second center point.