IMAGE PROCESSING DEVICE, CONTROL METHOD FOR AN IMAGE PROCESSING DEVICE, PROGRAM, AND INFORMATION STORAGE MEDIUM

• US 20130222647A1
• Filed: 04/25/2012
• Published: 08/29/2013
• Est. Priority Date: 06/27/2011
• Status: Active Grant

• Current Assignee: Konami Digital Entertainment Company Limited (Konami Holdings Corp.)
• Original Assignee: Konami Digital Entertainment Company Limited (Konami Holdings Corp.)
• Inventors: Ishihara, Akihiro, Ogawa, Kazuhiro, Matsumoto, Kazuya, Narita, Yoshihiko
• Granted Patent: US 8,866,848 B2
• Time in Patent Office: Days
• US Class Current: 348/239
• CPC Class Codes: G06T 15/503 Blending, e.g. for anti-aliasing
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: H04N 5/2621 Cameras specially adapted for the electronic generation of special effects during image pickup, e.g. digital cameras, camcorders, video cameras having integrated special effects capability
• CPC Class Codes: H04N 5/272 Means for inserting a foreground image in a background image, i.e. inlay, outlay

Abstract

Background object disposing means (88) disposes a background object (74) representing a background, which is photographed outside a target region (62) of a photographed image (60), on a virtual space (70). Subject object disposing means (90) disposes a subject object (74) between a viewpoint (72) and the background object (76) so that a position at which the subject object (76) is displayed to be superimposed on the background object (74) in a virtual space image (64), and a position of the target region (62) in the photographed image (60), correspond to each other. Composition target object disposing means (92) disposes a composition target object (78) representing a composition target, which is to be displayed to be combined with a real-world space (70) in the virtual space image (64), between the background object (74) and the subject object (76).

Claims

1. An image processing device, comprising at least one microprocessor configured to:

acquire a photographed image of a real-world space from a photographing device that photograph the real-world space;

cause a virtual space image representing a situation of a virtual space as viewed from a given viewpoint to be displayed on a display;

set a target region in the photographed image;

dispose a background object representing a background, which is photographed outside the target region of the photographed image, in the virtual space;

dispose a subject object representing a subject, which is photographed inside the target region of the photographed image, in the virtual space, and dispose the subject object between the given viewpoint and the background object so that a position at which the subject object is displayed to be superimposed on the background object in the virtual space image, and a position of the target region in the photographed image, correspond to each other; and

dispose a composition target object, which represents a composition target to be displayed to be combined with the real-world space in the virtual space image, between the background object and the subject object,

wherein the at least one microprocessor, in the virtual space image, display the composition target object to be superimposed on the background object, and display the subject object to be superimposed on the composition target object.

2. The image processing device according to claim 1,

wherein the at least one microprocessor acquire the photographed image from the photographing device, in a case where a photographing instruction operation is performed by the user,

wherein the at least one microprocessor cause a pre-photographing image representing the real-world space, which is acquired from the photographing device before the photographing instruction operation is performed by the user, to be displayed on the display, and guide, in the pre-photographing image, the user to the target region set in the photographed image, and

wherein the at least one microprocessor acquire the photographed image from the photographing device in a case where the user is guided to the target region and then performs the photographing instruction operation.

3. The image processing device according to claim 1,

wherein the at least one microprocessor allow the user to perform a changing instruction operation for a shape or the position of the target region set in the photographed image,

wherein the at least one microprocessor change, in a case where the changing instruction operation is performed by the user, the shape or the position of the target region based on the changing instruction operation,

wherein the at least one microprocessor determine a shape or the position of the subject object, based on the changed shape or position of the target region, and

wherein the at least one microprocessor determine a position or an action of the composition target object so that a whole or a portion of the composition target object is hidden by the determined subject object as viewed from the given viewpoint.

4. The image processing device according to claim 1,

wherein the subject object comprises a translucent polygon,

wherein the at least one microprocessor make the translucent polygon transmissive, representing a portion of the subject object that is photographed in a predetermined region in the target region, and

wherein the at least one microprocessor cause a portion of the composition target object, on which the transmissive translucent polygon is superimposed as viewed from the given viewpoint, to be displayed in the virtual space image.

5. The image processing device according to claim 1,

wherein the at least one microprocessor acquire motion data defining an action of the composition target object from a storage that store the virtual space and the motion data in association with each other, and

wherein the at least one microprocessor cause the composition target object to act in the virtual space, based on the motion data associated with the generated virtual space.

6. The image processing device according to claim 1,

wherein the at least one microprocessor guide the user to the subject to be photographed in the target region,

wherein the at least one microprocessor acquire motion data defining an action of the composition target object from a storage for store the subject and the motion data in association with each other, and

wherein the at least one microprocessor cause the composition target object to act in the virtual space, based on the motion data associated with the subject to which the user is guided.

7. A control method for an image processing device including at least one microprocessor, comprising:

acquiring, by the at least one microprocessor, a photographed image of a real-world space from a photographing device for photographing the real-world space;

causing, by the at least one microprocessor, a virtual space image representing a situation of a virtual space as viewed from a given viewpoint to be displayed on a display;

setting, by the at least one microprocessor, a target region in the photographed image;

disposing, by the at least one microprocessor, a background object representing a background, which is photographed outside the target region of the photographed image, in the virtual space;

disposing, by the at least one microprocessor, a subject object representing a subject, which is photographed inside the target region of the photographed image, in the virtual space, and disposing the subject object between the given viewpoint and the background object so that a position at which the subject object is displayed to be superimposed on the background object in the virtual space image, and a position of the target region in the photographed image, correspond to each other; and

disposing, by the at least one microprocessor, a composition target object, which represents a composition target to be displayed to be combined with the real-world space in the virtual space image, between the background object and the subject object,

wherein displaying, by the at least one microprocessor, in the virtual space image, the composition target object to be superimposed on the background object, and displaying the subject object to be superimposed on the composition target object.

8. A non-transitory computer readable information storage medium having a program recorded thereon, the program causing a computer to function as an image processing device, the image processing device comprising:

a image acquiring unit that acquires a photographed image of a real-world space from a photographing device for photographing the real-world space;

a display control unit that causes a virtual space image representing a situation of a virtual space as viewed from a given viewpoint to be displayed on a display means;

a target region setting unit that sets a target region in the photographed image;

a background object disposing unit that disposes a background object representing a background, which is photographed outside the target region of the photographed image, in the virtual space;

a subject object disposing unit that dispose a subject object representing a subject, which is photographed inside the target region of the photographed image, in the virtual space, the subject object disposing unit disposing the subject object between the given viewpoint and the background object so that a position at which the subject object is displayed to be superimposed on the background object in the virtual space image, and a position of the target region in the photographed image, correspond to each other; and

a composition target object disposing unit that disposes a composition target object, which represents a composition target to be displayed to be combined with the real-world space in the virtual space image, between the background object and the subject object,

wherein, in the virtual space image, the display control unit displays the composition target object to be superimposed on the background object, and displays the subject object to be superimposed on the composition target object.

9. (canceled)

10. An image processing device, comprising:

image acquiring means for acquiring a photographed image of a real-world space from photographing means for photographing the real-world space;

display control means for causing a virtual space image representing a situation of a virtual space as viewed from a given viewpoint to be displayed on display means;

target region setting means for setting a target region in the photographed image;

background object disposing means for disposing a background object representing a background, which is photographed outside the target region of the photographed image, in the virtual space;

subject object disposing means for disposing a subject object representing a subject, which is photographed inside the target region of the photographed image, in the virtual space, the subject object disposing means disposing the subject object between the given viewpoint and the background object so that a position at which the subject object is displayed to be superimposed on the background object in the virtual space image, and a position of the target region in the photographed image, correspond to each other; and

composition target object disposing means for disposing a composition target object, which represents a composition target to be displayed to be combined with the real-world space in the virtual space image, between the background object and the subject object,

wherein, in the virtual space image, the display control means displays the composition target object to be superimposed on the background object, and displays the subject object to be superimposed on the composition target object.

11. An image processing device, comprising:

a image acquiring unit that acquires a photographed image of a real-world space from a photographing device for photographing the real-world space;

a display control unit that causes a virtual space image representing a situation of a virtual space as viewed from a given viewpoint to be displayed on a display;

a target region unit that sets a target region in the photographed image;

a background object disposing unit that disposes a background object representing a background, which is photographed outside the target region of the photographed image, in the virtual space;

a subject object disposing unit that disposes a subject object representing a subject, which is photographed inside the target region of the photographed image, in the virtual space, the subject object disposing unit disposing the subject object between the given viewpoint and the background object so that a position at which the subject object is displayed to be superimposed on the background object in the virtual space image, and a position of the target region in the photographed image, correspond to each other; and

a composition target object disposing unit that disposes a composition target object, which represents a composition target to be displayed to be combined with the real-world space in the virtual space image, between the background object and the subject object,

wherein, in the virtual space image, the display control unit displays the composition target object to be superimposed on the background object, and displays the subject object to be superimposed on the composition target object.