Image Processing Device, Image Processing Method, and Program

• US 20110187746A1
• Filed: 01/05/2011
• Published: 08/04/2011
• Est. Priority Date: 02/02/2010
• Status: Active Grant

• Current Assignee: Sony Corporation (Sony Group Corp.)
• Original Assignee: Sony Corporation (Sony Group Corp.)
• Inventors: Fukuchi, Masaki, Matsuda, Kouichi, Oi, Kenichiro, Guo, Jingjing, SUTO, Yasuhiro
• Granted Patent: US 8,896,628 B2
• Time in Patent Office: Days
• US Class Current: 345/634
• CPC Class Codes: G02B 2027/014 comprising information/image processing systems
• CPC Class Codes: G02B 27/0172 characterised by optical features
• CPC Class Codes: G06T 11/60 Editing figures and text; Combining figures or text
• CPC Class Codes: G06T 17/00 Three dimensional [3D] modelling, e.g. data description of 3D objects
• CPC Class Codes: G06T 19/003 Navigation within 3D models or images
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G06T 2210/61 Scene description
• CPC Class Codes: G06V 20/20 in augmented reality scenes
• CPC Class Codes: G06V 20/52 Surveillance or monitoring of activities, e.g. for recognising suspicious objects recognising microscopic objects G06V20/69
• CPC Class Codes: G09G 5/00 Control arrangements or circuits for visual indicators common to cathode-ray tube indicators and other visual indicators image data processing or generation, in general G06T

Abstract

There is provided an image processing device including: a data storage unit storing feature data indicating a feature of appearance of one or more physical objects; an environment map building unit for building an environment map based on an input image obtained by imaging a real space and the feature data, the environment map representing a position of a physical object present in the real space; a control unit for acquiring procedure data for a set of procedures of operation to be performed in the real space, the procedure data defining a correspondence between a direction for each procedure and position information designating a position at which the direction is to be displayed; and a superimposing unit for generating an output image by superimposing the direction for each procedure at a position in the input image determined based on the environment map and the position information, using the procedure data.

Claims

1. An image processing device comprising:

a data storage unit having feature data stored therein, the feature data indicating a feature of appearance of one or more physical objects;

an environment map building unit for building an environment map based on an input image obtained by imaging a real space using an imaging device and the feature data stored in the data storage unit, the environment map representing a position of a physical object present in the real space;

a control unit for acquiring procedure data for a set of procedures of operation to be performed in the real space, the procedure data defining a correspondence between a direction for each procedure and position information designating a position at which the direction is to be displayed; and

a superimposing unit for generating an output image by superimposing the direction for each procedure included in the set of procedures at a position in the input image determined based on the environment map and the position information, using the procedure data acquired by the control unit.

2. The image processing device according to claim 1,

wherein the position information designates a position in the environment map at which the direction is to be displayed, by specifying a physical object related to the direction for each procedure.

3. The image processing device according to claim 1,

wherein the procedure data defines a further correspondence between the direction for each procedure and a condition for progressing displaying of each direction.

4. The image processing device according to claim 3,

wherein the condition for progressing displaying of each direction includes a condition according to a position or a posture of a physical object represented by the environment map.

5. The image processing device according to claim 3,

wherein the control unit controls displaying of the direction for each procedure included in the set of procedures according to the procedure data.

6. The image processing device according to claim 1,

wherein the image processing device further comprises a detection unit for dynamically detecting a position in the real space of the imaging device based on the input image and the feature data, and

the position in the input image at which the direction for each procedure is to be superimposed is determined according to the position in the real space of the imaging device detected by the detection unit.

7. An image processing method in an image processing device including a storage medium having feature data stored therein, the feature data indicating a feature of appearance of one or more physical objects, the method comprising the steps of:

acquiring an input image obtained by imaging a real space using an imaging device;

building an environment map based on the input image and the feature data, the environment map representing a position of a physical object present in the real space;

acquiring procedure data for a set of procedures of operation to be performed in the real space, the procedure data defining a correspondence between a direction for each procedure and position information designating a position at which the direction is to be displayed; and

generating an output image by superimposing the direction for each procedure included in the set of procedures at a position in the input image determined based on the environment map and the position information, using the acquired procedure data.

8. A program for causing a computer, which controls an image processing device including a storage medium having feature data stored therein, the feature data indicating a feature of appearance of one or more physical objects, to function as:

an environment map building unit for building an environment map based on an input image obtained by imaging a real space using an imaging device and the feature data, the environment map representing a position of a physical object present in the real space;

a control unit for acquiring procedure data for a set of procedures of operation to be performed in the real space, the procedure data defining a correspondence between a direction for each procedure and position information designating a position at which the direction is to be displayed; and

a superimposing unit for generating an output image by superimposing the direction for each procedure included in the set of procedures at a position in the input image determined based on the environment map and the position information, using the procedure data acquired by the control unit.