Image combination device, image combination method, image combination program, and recording medium containing the image combination program

• US 20060078224A1
• Filed: 07/03/2003
• Published: 04/13/2006
• Est. Priority Date: 08/09/2002
• Status: Active Grant

• Current Assignee: Sharp Kabushiki Kaisha (Hon Hai Precision Industry Co., Ltd.)
• Original Assignee: Sharp Kabushiki Kaisha (Hon Hai Precision Industry Co., Ltd.)
• Inventors: Hirosawa, Masashi
• Granted Patent: US 7,623,733 B2
• Time in Patent Office: Days
• US Class Current: 382/284
• CPC Class Codes: H04N 23/73 by influencing the exposure time
• CPC Class Codes: H04N 5/272 Means for inserting a foreground image in a background image, i.e. inlay, outlay

Abstract

An image combination device acquires a first object image, a second object image, and a background image by first object image acquiring means (1), second object image acquiring means (3), and background image acquiring means (2) and by using one of the three images as a standard image, generates two corrected images by background correction amount calculating means (4) and corrected image generating means (5), so that background portions of the two images correspond. Superimposed image generating means (9) generates an image by superimposing the standard image and one or two of the corrected images. According to this arrangement, since the backgrounds of the superimposed images correspond to each other, the first object and the second object, which have been photographed separately, can be combined without causing an impression that these objects are combined to the background artificially.

Claims

1. An image combination device, comprising:

amount of background correction calculating means for calculating an amount of background correction or reading out the amount of background correction after the amount of background correction is calculated and recorded, the background correction being performed among (i) a background image, which is an image of a background, (ii) a first object image, which includes at least a part of the background and a first object, and (iii) a second object image, which includes at least a part of the background and a second object, the amount of background correction being one or a combination of relative amounts including an amount of movement, an amount of rotation, a rate of expansion or reduction, and an amount of distortion correction, with respect to a background; and

superimposed image generating means for generating a superimposed image by using one of the background image, the first object image, and the second object image as a standard image, correcting the other two images by the amount of correction obtained from the amount of background correction calculating means, the other two images being corrected so that backgrounds, other than objects, of the other two images correspond to the standard image at least partially, and superimposing the standard image and one or both of the other two images.

2. The image combination device as set forth in claim 1, further comprising:

image pickup means for picking up an image of an object or a scene,

the background image, the first object image, or the second object image being generated based on an output of the image pickup means.

3. The image combination device as set forth in claim 2, wherein:

the first object image or the second object image that is photographed earlier than the other is used as the standard image.

4. The image combination device as set forth in claim 3, wherein:

the background image is photographed immediately before or immediately after the standard image is photographed.

5. The image combination device as set forth in claim 1, wherein:

the superimposed image generating means superimposes the standard image and the other one or two corrected images respectively at predetermined transmittances.

6. The image combination device as set forth in claim 1, wherein:

the superimposed image generating means generates a difference image from the standard image and the other one or two corrected images, and a region in the difference image that has a difference is generated as an image having a pixel value that is different from an original pixel value.

7. The image combination device as set forth in claim 1, further comprising:

object region extracting means for extracting a region of the first object and a region of the second object from a difference image generated from the standard image and the other one or two corrected images,

the superimposed image generating means superimposing the standard image and the regions of the one or two corrected images obtained from the object region extracting means, instead of superimposing the standard image and the one or two corrected images.

8. The image combination device as set forth in claim 7, further comprising:

overlap detecting means for detecting overlap between the region of the first object and the region of the second object obtained from the object region extracting means.

9. The image combination device as set forth in claim 8, further comprising:

overlap warning means for warning the user and/or the object that there is overlap, when overlap is detected by the overlap detecting means.

10. The image combination device as set forth in claim 8, further comprising:

shutter release timing notifying means for notifying the user and/or the object that there is no overlap, when no overlap is detected by the overlap detecting means.

11. The image combination device as set forth in claim 8, further comprising:

image pickup means for picking up an image of an object or a scene; and

automatic shutter releasing means for generating an instruction when no overlap is detected by the overlap detecting means, the instruction instructing that the image obtained from the image pickup means be recorded as the background image, the first object image, or the second object image.

12. The image combination device as set forth in claim 8, further comprising:

image pickup means for picking up an image of an object or a scene; and

automatic shutter releasing means for generating an instruction when overlap is detected by the overlap detecting means, the instruction instructing that the image obtained from the image pickup means should not be recorded as the background image, the first object image, or the second object image.

13. The image combination device as set forth in claim 8, wherein:

the overlap detecting means extracts an overlap region where the region of the first object and the region of the second object overlap.

14. The image combination device as set forth in claim 13, wherein:

the superimposed image generating means generates the superimposed image in such a manner that the overlap region extracted by the overlap detecting means has a pixel value that is different from an original pixel value.

15. The image combination device as set forth in any one of claims 8 to 14, further comprising:

overlap prevention method calculating means for calculating a position or a direction of the position of the first object or the second object when overlap is detected by the overlap detecting means, the position being a position at which the overlap is reduced; and

overlap prevention method notifying means for notifying the user and/or the object of the position or the direction of the position of the first object or the second object, the position or the direction of the position being obtained from the overlap prevention method calculating means.

31. An image combination program for causing a computer to function as each means of the image combination device as set forth in claims 1 and 17.

34. A recording medium containing the image combination program as set forth in claim 31.

16. An image combination method, comprising:

amount of background correction calculating step, in which an amount of background correction is calculated, or the amount of background correction is read out after the amount of background correction is calculated and recorded, the background correction being performed among (i) a background image, which is an image of a background, (ii) a first object image, which includes at least a part of the background and a first object, and (iii) a second object image, which includes at least a part of the background and a second object, the amount of background correction being one or a combination of relative amounts including an amount of movement, an amount of rotation, a rate of expansion or reduction, and an amount of distortion correction, with respect to a background; and

superimposed image generating step, in which a superimposed image is generated by using one of the background image, the first object image, and the second object image as a standard image, correcting the other two images by the amount of correction obtained in the amount of background correction calculating step, the other two images being corrected so that backgrounds, other than objects, of the other two images correspond to the standard image at least partially, and superimposing the standard image and one or both of the other two images.

33. An image combination program for causing a computer to execute each step of the image combination method as set forth in claim 16 or claim 30.

35. A recording medium containing the image combination program as set forth in claim 33.

36. A recording medium containing the image combination program as set forth in claim 33.

17. An image combination device, comprising:

amount of background correction calculating means for calculating an amount of background correction or reading out the amount of background correction after the amount of background correction is calculated, the background correction being performed among (i) a first object image, which includes a background and a first object, and (ii) a second object image, which includes at least a part of the background and a second object, the amount of background correction being one or a combination of relative amounts including an amount of movement, an amount of rotation, a rate of expansion or reduction, and an amount of distortion correction, with respect to a background; and

superimposed image generating means for generating a superimposed image by using one of the first object image or the second object image as a standard image, correcting the other of the first object image or the second object image by the amount of correction obtained from the amount of background correction calculating means, so that a background portion, other than the object, of the other of the first object image or the second object image corresponds to the standard image at least partially, and superimposing the standard image and a corrected image.

18. The image combination device as set forth in claim 17, further comprising:

image pickup means for picking up an image of an object or a scene,

the first object image or the second object image being generated based on an output of the image pickup means.

19. The image combination device as set forth in claim 18, wherein:

the first object image or the second object image that is photographed later than the other is used as the standard image.

20. The image combination device as set forth in claim 17, wherein:

the superimposed image generating means superimposes the standard image and the corrected image respectively at predetermined transmittances.

21. The image combination device as set forth in claim 17, wherein:

the superimposed image generating means generates a difference image from the standard image and the corrected image, and a region in the difference image that has a difference is generated as an image having a pixel value that is different from an original pixel value.

22. The image combination device as set forth in claim 17, further comprising:

object region extracting means for extracting a region of the first object and a region of the second object from a difference image generated from the standard image and the corrected image,

the superimposed image generating means superimposing (a) the standard image or the corrected image and (b) images within the regions obtained from the object region extracting means, instead of superimposing the standard image and the corrected image.

23. The image combination device as set forth in claim 22, wherein:

the object region extracting means extracts an image within the region of the first object and an image within the region of the second object from the first object image or a corrected first object image, extracts an image within the region of the first object and an image within the region of the second object from the second object image or a corrected second object image, and discriminates between an image of the first object and an image of the second object by using skin color as a benchmark.

24. The image combination device as set forth in claim 22, wherein:

the object region extracting means extracts an image within the region of the first object and an image within the region of the second object from the first object image or a corrected first object image, extracts an image within the region of the first object and an image within the region of the second object from the second object image or a corrected second object image, and discriminates between an image of the first object and an image of the second object by using, as a benchmark, a characteristic of an image outside each region.

25. The image combination device as set forth in claim 22, further comprising:

overlap detecting means that judges that the region of the first object and the region of the second object overlap, when the number of regions of the first object or the second object obtained from the object region extracting means does not correspond to a value set as the number of objects to be combined.

26. The image combination device as set forth in claim 25, further comprising:

overlap warning means for warning the user and/or the object that there is overlap, when overlap is detected by the overlap detecting means.

27. The image combination device as set forth in claim 25, further comprising:

shutter release timing notifying means for notifying the user and/or the object that there is no overlap, when no overlap is detected by the overlap detecting means.

28. The image combination device as set forth in claim 25, further comprising:

image pickup means for picking up an image of an object or a scene; and

automatic shutter releasing means for generating an instruction when no overlap is detected by the overlap detecting means, the instruction instructing that the image obtained from the image pickup means be recorded as the first object image or the second object image.

29. The image combination device as set forth in claim 25, further comprising:

image pickup means for picking up an image of an object or a scene; and

automatic shutter releasing means for generating an instruction when overlap is detected by the overlap detecting means, the instruction instructing that the image obtained from the image pickup means should not be recorded as the first object image or the second object image.

30. An image combination method, comprising:

amount of background correction calculating step, in which an amount of background correction is calculated, or the amount of background correction is read out after the amount of background correction is calculated and recorded, the background correction being performed between (i) a first object image, which includes a background and a first object and (ii) a second object image, which includes at least a part of the background and a second object, the amount of background correction being one or a combination of relative amounts including an amount of movement, an amount of rotation, a rate of expansion or reduction, and an amount of distortion correction, with respect to a background; and

superimposed image generating step, in which a superimposed image is generated by using one of the first object image or the second object image as a standard image, correcting the other of the first object image or the second object image by the amount of correction obtained from the amount of background correction calculating means, so that a background portion, other than the object, of the other image corresponds to the standard image at least partially, and superimposing the standard image and the corrected image.

32. (canceled)