IMAGE PROCESSING SYSTEM, METHOD AND APPARATUS, AND COMPUTER-READABLE MEDIUM RECORDING IMAGE PROCESSING PROGRAM

• US 20120218300A1
• Filed: 02/27/2012
• Published: 08/30/2012
• Est. Priority Date: 02/25/2011
• Status: Active Grant

• Current Assignee: Nintendo Company Limited
• Original Assignee: Nintendo Company Limited
• Inventors: HAYAKAWA, Takeshi
• Granted Patent: US 9,030,493 B2
• Time in Patent Office: Days
• US Class Current: 345/633
• CPC Class Codes: A63F 13/213 comprising photodetecting means, e.g. cameras, photodiodes or infrared cells A63F13/219, A63F13/655 take precedence
• CPC Class Codes: A63F 13/26 having at least one additional display device, e.g. on the game controller or outside a game booth
• CPC Class Codes: A63F 13/42 by mapping the input signals into game commands, e.g. mapping the displacement of a stylus on a touch screen to the steering angle of a virtual vehicle
• CPC Class Codes: A63F 13/52 involving aspects of the displayed game scene
• CPC Class Codes: A63F 13/5255 according to dedicated instructions from a player, e.g. using a secondary joystick to rotate the camera around a player's character
• CPC Class Codes: A63F 13/655 by importing photos, e.g. of the player
• CPC Class Codes: A63F 13/92 Video game devices specially adapted to be hand-held while playing
• CPC Class Codes: A63F 2300/1093 using visible light
• CPC Class Codes: A63F 2300/6045 for mapping control signals received from the input arrangement into game commands
• CPC Class Codes: A63F 2300/66 for rendering three dimensional images
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00

Abstract

An example image processing apparatus has a captured image acquisition unit for acquiring a captured image captured by an imaging device, a feature detection unit for detecting the markers from the captured image, a reference acquisition unit for acquiring, based on each of the detected markers, a coordinate system serving as a reference indicating a position and attitude in a space, and a relative relation information acquisition unit for acquiring, based on the captured image in which a plurality of the markers are detected, relative relation information indicating a relative relation in position and attitude of a plurality of coordinate systems acquired for the respective markers.

Claims

1. An image processing system comprising:

a component marked with a feature whose position and attitude relative to an imaging device can be specified by being imaged by the imaging device; and

an image processing apparatus connected to the imaging device,

the image processing apparatus comprising;

a captured image acquisition unit for acquiring a captured image captured by the imaging device;

a feature detection unit for detecting the feature from the captured image;

a reference acquisition unit for acquiring a position/attitude reference based on the detected feature, the reference being used as a reference for indicating a position and attitude in a space imaged by the imaging device; and

a relative relation information acquisition unit for acquiring, based on the captured image from which the feature is detected in plurality, relative relation information indicating a relative relation in position and attitude among the plurality of position/attitude references acquired for the respective features.

2. The image processing system according to claim 1, wherein the image processing apparatus further comprises an estimation unit for estimating, based on the relative relation information and one of the position/attitude references related to the relative relation information, another one of the position/attitude references.

3. The image processing system according to claim 1, wherein the image processing apparatus further comprises a storage unit for storing the relative relation information.

4. The image processing system according to claim 3, wherein the image processing apparatus further comprises:

a determination unit for determining whether or not the relative relation information acquired by the relative relation information acquisition unit is relative relation information corresponding to the relative relation information stored in the storage unit; and

an update unit for updating the stored relative relation information using the acquired relative relation information, when it is determined by the determination unit that the acquired relative relation information is relative relation information corresponding to the stored relative relation information.

5. The image processing system according to claim 4, wherein the image processing apparatus further comprises an addition unit for storing the acquired relative relation information in the storage unit when it is not determined by the determination unit that the acquired relative relation information is relative relation information corresponding to the stored relative relation information.

6. The image processing system according to claim 3, wherein the image processing apparatus further comprises a deletion unit for deleting the relative relation information satisfying a predetermined condition from the storage unit.

7. The image processing system according to claim 1, wherein the image processing apparatus further comprises a rendering unit for rendering a virtual space image by arranging a virtual camera in a virtual space, arranging, in the virtual space, a virtual object whose position and attitude are determined according to the position/attitude reference, and generating an image of the virtual space as viewed from the virtual camera.

8. The image processing system according to claim 7, wherein the image processing apparatus further comprises a display control unit for generating a composite image by overlapping the image of the virtual space with the captured image and causing a display device to display the composite image.

9. The image processing system according to claim 1, wherein the image processing apparatus further comprises a processing unit for performing predetermined processing so that time in the virtual space advances with passage of real time.

10. An image processing method comprising:

an captured image acquisition of acquiring a captured image captured by an imaging device;

a feature detection of detecting, from the captured image, a feature whose position and attitude relative to the imaging device in a space imaged by the imaging device can be specified;

a reference acquisition of acquiring, based on the detected feature, a position/attitude reference which is used as a reference indicating a position and attitude in the space imaged by the imaging device; and

a relative relation information acquisition of acquiring, based on the captured image from which the feature is detected in plurality, relative relation information indicating a relative relation in position and attitude among the plurality of position/attitude references acquired for the respective features.

11. An image processing apparatus comprising:

a captured image acquisition unit for acquiring a captured image captured by an imaging device;

a feature detection unit for detecting, from the captured image, a feature whose position and attitude relative to the imaging device in a space imaged by the imaging device can be specified;

a reference acquisition unit for acquiring, based on the detected feature, a position/attitude reference which is used as a reference indicating a position and attitude in the space imaged by the imaging device; and

a relative relation information acquisition unit for acquiring, based on the captured image from which the feature is detected in plurality, relative relation information indicating a relative relation in position and attitude among the plurality of position/attitude references acquired for the respective features.

12. A computer-readable medium on which an image processing program is recorded, the image processing program causing a computer to function as:

a captured image acquisition unit for acquiring a captured image captured by an imaging device;

a feature detection unit for detecting, from the captured image, a feature whose position and attitude relative to the imaging device in a space imaged by the imaging device can be specified;

a reference acquisition unit for acquiring, based on the detected feature, a position/attitude reference which is used as a reference indicating a position and attitude in the space imaged by the imaging device; and

a relative relation information acquisition unit for acquiring, based on the captured image from which the feature is detected in plurality, relative relation information indicating a relative relation in position and attitude among the plurality of position/attitude references acquired for the respective features.