Image processing device, image processing method, and program

• US 9,754,418 B2
• Filed: 09/29/2015
• Issued: 09/05/2017
• Est. Priority Date: 02/02/2010
• Status: Active Grant

• Current Assignee: Sony Corporation (Sony Group Corp.)
• Original Assignee: Sony Corporation (Sony Group Corp.)
• Inventors: Suto, Yasuhiro, Fukuchi, Masaki, Ooi, Kenichirou, Guo, Jingjing, Matsuda, Kouichi
• Primary Examiner(s): Hoang, Phi
• Application Number:
  US14/869,063
• Publication Number:
  US 20160035141A1
• Time in Patent Office: 707 Days
• CPC Class Codes: G02B 2027/014 comprising information/image processing systems
• CPC Class Codes: G02B 27/0172 characterised by optical features
• CPC Class Codes: G06T 11/60 Editing figures and text; Combining figures or text
• CPC Class Codes: G06T 17/00 Three dimensional [3D] modelling, e.g. data description of 3D objects
• CPC Class Codes: G06T 19/003 Navigation within 3D models or images
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G06T 2210/61 Scene description
• CPC Class Codes: G06V 20/20 in augmented reality scenes
• CPC Class Codes: G06V 20/52 Surveillance or monitoring of activities, e.g. for recognising suspicious objects recognising microscopic objects G06V20/69
• CPC Class Codes: G09G 5/00 Control arrangements or circuits for visual indicators common to cathode-ray tube indicators and other visual indicators image data processing or generation, in general G06T

Abstract

There is provided an image processing device including: a data storage unit storing feature data indicating a feature of appearance of one or more physical objects; an environment map building unit for building an environment map based on an input image obtained by imaging a real space and the feature data, the environment map representing a position of a physical object present in the real space; a control unit for acquiring procedure data for a set of procedures of operation to be performed in the real space, the procedure data defining a correspondence between a direction for each procedure and position information designating a position at which the direction is to be displayed; and a superimposing unit for generating an output image by superimposing the direction for each procedure at a position in the input image determined based on the environment map and the position information, using the procedure data.

Claims

1. An image processing device comprising:

circuitry configured to

identify an object present in a real space, and

control a displaying of a series of directions comprising more than two procedures, the more than two procedures being defined by an order to be read, and at least one of which is associated with the object present in the real space and displayed at a timing defined by a progress condition corresponding to the at least one procedure,

wherein the progress condition is defined by one of a state of the object present in the real space, a passage of a given time, and an external event.

2. The image processing device according to claim 1,

wherein a direction of the at least one procedure is displayed in relation to the object or to a vicinity the object.

3. The image processing device according to claim 1,

wherein the object corresponds to a related physical object that is associated with the at least one procedure.

4. The image processing device according to claim 3,

wherein the related physical object is blank when associated with a procedure that is not associated with any physical object.

5. The image processing device according to claim 1,

wherein the series of directions are displayed to a user in a transit station.

6. The image processing device according to claim 1,

wherein the series of directions are displayed to a user in a commercial facility.

7. The image processing device according to claim 1,

wherein the series of directions are displayed to a user to instruct the user of a transaction process.

8. The image processing device according to claim 1,

wherein the series of directions is displayed to be superimposed upon or in the vicinity of the object in the real space or a representation of the object.

9. An image processing method comprising:

identifying an object present in a real space; and

displaying a series of directions comprising more than two procedures, the more than two procedures being defined by an order to be read, and at least one of which is associated with the object present in the real space and displayed at a timing defined by a progress condition corresponding to the at least one procedure,

wherein the progress condition is defined by one of a state of the object present in the real space, a passage of a given time, and an external event.

10. The image processing method according to claim 9,

wherein a direction of the at least one procedure is displayed in relation to the object or to a vicinity the object.

11. The image processing method according to claim 9,

wherein the object corresponds to a related physical object that is associated with the at least one procedure.

12. The image processing method according to claim 11,

wherein the related physical object is blank when associated with a procedure that is not associated with any physical object.

13. The image processing method according to claim 9,

wherein the series of directions are displayed to a user in a transit station.

14. The image processing method according to claim 9,

wherein the series of directions are displayed to a user in a commercial facility.

15. The image processing method according to claim 9,

wherein the series of directions are displayed to a user to instruct the user of a transaction process.

16. The image processing method according to claim 9,

wherein the series of directions is displayed to be superimposed upon the object in the real space or in the vicinity of the object in the real space, or superimposed upon a representation of the object or in the vicinity of the representation of the object.

17. A non-transitory computer-readable medium having embodied thereon a program, which when executed by a computer causes the computer to execute a method, the method comprising:

identifying an object present in a real space; and

displaying a series of directions comprising more than two procedures, the more than two procedures being defined by an order to be read, and at least one of which is associated with the object present in the real space and displayed at a timing defined by a progress condition corresponding to the at least one procedure,

wherein the progress condition is defined by one of a state of the object present in the real space, a passage of a given time, and an external event.