Image combination device, image combination method, image combination program, and recording medium for combining images having at least partially same background

• US 7,623,733 B2
• Filed: 07/03/2003
• Issued: 11/24/2009
• Est. Priority Date: 08/09/2002
• Status: Expired due to Fees

• Current Assignee: Sharp Kabushiki Kaisha (Hon Hai Precision Industry Co., Ltd.)
• Original Assignee: Sharp Kabushiki Kaisha (Hon Hai Precision Industry Co., Ltd.)
• Inventors: Hirosawa, Masashi
• Primary Examiner(s): Mehta; Bhavesh M
• Assistant Examiner(s): ZARKA, DAVID PETER
• Application Number:
  US10/523,770
• Publication Number:
  US 20060078224A1
• Time in Patent Office: 2,336 Days
• Field of Search: 382/284
• US Class Current: 382/284
• CPC Class Codes: H04N 23/73 by influencing the exposure time
• CPC Class Codes: H04N 5/272 Means for inserting a foreground image in a background image, i.e. inlay, outlay

Abstract

An image combination device acquires a first object image, a second object image, and a background image by first object image acquiring section (1), second object image acquiring section (3), and background image acquiring section (2) and by using one of the three images as a standard image, generates two corrected images by background correction amount calculating section (4) and corrected image generating section (5), so that background portions of the two images correspond. Superimposed image generating section (9) generates an image by superimposing the standard image and one or two of the corrected images. According to this arrangement, since the backgrounds of the superimposed images correspond to each other, the first object and the second object, which have been photographed separately, can be combined without causing an impression that these objects are combined to the background artificially.

Claims

1. An image combination device, comprising:

amount of background correction calculating means for calculating an amount of background correction or reading out the amount of background correction after the amount of background correction is calculated and recorded, the amount of background correction being performed among (i) a background image, which is an image of a background, (ii) a first object image, which includes at least a part of the background and a first object, and (iii) a second object image, which includes at least a part of the background and a second object, the amount of background correction being one or a combination of relative amounts including an amount of movement, an amount of rotation, a rate of expansion or reduction, and an amount of distortion correction, with respect to a background;

superimposed image generating means for generating a superimposed image by using one of the background image, the first object image, and the second object image as a standard image, correcting the other two images by the amount of background correction obtained from the amount of background correction calculating means, the other two images being corrected so that backgrounds, other than objects, of the other two images correspond to the standard image at least partially, and superimposing the standard image and one or both of the other two images; and

object region extracting means for extracting a region of the first object and a region of the second object from a difference image generated from the standard image and the other one or two corrected images,

the superimposed image generating means superimposing the standard image and the regions of the one or two corrected images obtained from the object region extracting means, instead of superimposing the standard image and the one or two corrected images.

2. The image combination device as set forth in claim 1, further comprising:

image pickup means for picking up an image of an object or a scene,

the background image, the first object image, or the second object image being generated based on an output of the image pickup means.

3. The image combination device as set forth in claim 2, wherein:

either of the first object image or the second object image that is photographed earlier is used as the standard image.

4. The image combination device as set forth in claim 3, wherein:

the background image is photographed immediately before or immediately after the standard image is photographed.

5. The image combination device as set forth in claim 1, wherein:

the superimposed image generating means superimposes the standard image and the other one or two corrected images respectively at predetermined transmittances.

6. The image combination device as set forth in claim 1, wherein:

the superimposed image generating means generates a difference image from the standard image and the other one or two corrected images, and a region in the difference image that has a difference is generated as an image having a pixel value that is different from an original pixel value.

7. The image combination device as set forth in claim 1, further comprising:

overlap detecting means for detecting overlap between the region of the first object and the region of the second object obtained from the object region extracting means.

8. The image combination device as set forth in claim 7, further comprising:

overlap warning means for warning a user and/or the object that there is overlap, when overlap is detected by the overlap detecting means.

9. The image combination device as set forth in claim 7, further comprising:

shutter release timing notifying means for notifying the user and/or the object that there is no overlap, when no overlap is detected by the overlap detecting means.

10. The image combination device as set forth in claim 7, further comprising:

image pickup means for picking up an image of an object or a scene; and

automatic shutter releasing means for generating an instruction when no overlap is detected by the overlap detecting means, the instruction instructing that the image obtained from the image pickup means be recorded as the background image, the first object image, or the second object image.

11. The image combination device as set forth in claim 7, further comprising:

image pickup means for picking up an image of an object or a scene; and

automatic shutter releasing means for generating an instruction when overlap is detected by the overlap detecting means, the instruction instructing that the image obtained from the image pickup means should not be recorded as the background image, the first object image, or the second object image.

12. The image combination device as set forth in claim 7, wherein:

the overlap detecting means extracts an overlap region where the region of the first object and the region of the second object overlap.

13. The image combination device as set forth in claim 12, wherein:

the superimposed image generating means generates the superimposed image in such a manner that the overlap region extracted by the overlap detecting means has a pixel value that is different from an original pixel value.

14. The image combination device as set forth in any one of claims 7 to 13, further comprising:

overlap prevention method calculating means for calculating a position or a direction of the position of the first object or the second object when overlap is detected by the overlap detecting means, the position being a position at which the overlap is reduced; and

overlap prevention method notifying means for notifying the user and/or the object of the position or the direction of the position of the first object or the second object, the position or the direction of the position being obtained from the overlap prevention method calculating means.

27. An image combination program encoded within a computer-readable medium, wherein the device set forth in claim 1 includes said computer-readable medium to, when executed by a computer, perform the functions set forth in claim 1.

15. An image combination device, comprising:

amount of background correction calculating means for calculating an amount of background correction or reading out the amount of background correction after the amount of background correction is calculated, the amount of background correction being performed among (i) a first object image, which includes a background and a first object, and (ii) a second object image, which includes at least a part of the background and a second object, the amount of background correction being one or a combination of relative amounts including an amount of movement, an amount of rotation, a rate of expansion or reduction, and an amount of distortion correction, with respect to a background;

superimposed image generating means for generating a superimposed image by using one of the first object image or the second object image as a standard image, correcting the other of the first object image or the second object image by the amount of background correction obtained from the amount of background correction calculating means, so that a background portion, other than the object, of the other of the first object image or the second object image corresponds to the standard image at least partially, and superimposing the standard image and a corrected image; and

object region extracting means for extracting a region of the first object and a region of the second object from a difference image generated from the standard image and the corrected image,

the superimposed image generating means superimposing (a) the standard image or the corrected image and (b) images within the regions obtained from the object region extracting means, instead of superimposing the standard image and the corrected image.

16. The image combination device as set forth in claim 15, further comprising:

image pickup means for picking up an image of an object or a scene,

the first object image or the second object image being generated based on an output of the image pickup means.

17. The image combination device as set forth in claim 16, wherein:

the first object image or the second object image that is photographed later than the other is used as the standard image.

18. The image combination device as set forth in claim 15, wherein:

the superimposed image generating means superimposes the standard image and the corrected image respectively at predetermined transmittances.

19. The image combination device as set forth in claim 15, wherein:

the superimposed image generating means generates a difference image from the standard image and the corrected image, and a region in the difference image that has a difference is generated as an image having a pixel value that is different from an original pixel value.

20. The image combination device as set forth in claim 15, wherein:

the object region extracting means extracts an image within the region of the first object and an image within the region of the second object from the first object image or a corrected first object image, extracts an image within the region of the first object and an image within the region of the second object from the second object image or a corrected second object image, and discriminates between an image of the first object and an image of the second object by using skin color as a benchmark.

21. The image combination device as set forth in claim 15, wherein:

the object region extracting means extracts an image within the region of the first object and an image within the region of the second object from the first object image or a corrected first object image, extracts an image within the region of the first object and an image within the region of the second object from the second object image or a corrected second object image, and discriminates between an image of the first object and an image of the second object by using, as a benchmark, a characteristic of an image outside each region.

22. The image combination device as set forth in claim 15, further comprising:

overlap detecting means that judges that the region of the first object and the region of the second object overlap, when the number of regions of the first object or the second object obtained from the object region extracting means does not correspond to a value set as the number of objects to be combined.

23. The image combination device as set forth in claim 22, further comprising:

overlap warning means for warning the user and/or the object that there is overlap, when overlap is detected by the overlap detecting means.

24. The image combination device as set forth in claim 22, further comprising:

shutter release timing notifying means for notifying the user and/or the object that there is no overlap, when no overlap is detected by the overlap detecting means.

25. The image combination device as set forth in claim 22, further comprising:

image pickup means for picking up an image of an object or a scene; and

automatic shutter releasing means for generating an instruction when no overlap is detected by the overlap detecting means, the instruction instructing that the image obtained from the image pickup means be recorded as the first object image or the second object image.

26. The image combination device as set forth in claim 22, further comprising:

image pickup means for picking up an image of an object or a scene; and

automatic shutter releasing means for generating an instruction when overlap is detected by the overlap detecting means, the instruction instructing that the image obtained from the image pickup means should not be recorded as the first object image or the second object image.

28. An image combination device, comprising:

amount of background correction calculating means for calculating an amount of background correction or reading out the amount of background correction after the amount of background correction is calculated and recorded, the amount of background correction being performed among (i) a background image, which is an image of a background, (ii) a first object image, which includes at least a part of the background and a first object, and (iii) a second object image, which includes at least a part of the background and a second object, the amount of background correction being one or a combination of relative amounts including an amount of movement, an amount of rotation, a rate of expansion or reduction, and an amount of distortion correction, with respect to a background; and

superimposed image generating means for generating a superimposed image by using one of the background image, the first object image, and the second object image as a standard image, correcting the other two images by the amount of background correction obtained from the amount of background correction calculating means, the other two images being corrected so that backgrounds, other than objects, of the other two images correspond to the standard image at least partially, and superimposing the standard image and one or both of the other two images, wherein;

the superimposed image generating means generates a difference image from the standard image and the other one or two corrected images, and a region in the difference image that has a difference is generated as an image having a pixel value that is different from an original pixel value.

29. An image combination device, comprising:

amount of background correction calculating means for calculating an amount of background correction or reading out the amount of background correction after the amount of background correction is calculated, the amount of background correction being performed among (i) a first object image, which includes a background and a first object, and (ii) a second object image, which includes at least a part of the background and a second object, the amount of background correction being one or a combination of relative amounts including an amount of movement, an amount of rotation, a rate of expansion or reduction, and an amount of distortion correction, with respect to a background; and

superimposed image generating means for generating a superimposed image by using one of the first object image or the second object image as a standard image, correcting the other of the first object image or the second object image by the amount of background correction obtained from the amount of background correction calculating means, so that a background portion, other than the object, of the other of the first object image or the second object image corresponds to the standard image at least partially, and superimposing the standard image and a corrected image, wherein;

the superimposed image generating means generates a difference image from the standard image and the corrected image, and a region in the difference image that has a difference is generated as an image having a pixel value that is different from an original pixel value.