Image processing device, object selection method and program

• US 9,288,423 B2
• Filed: 03/13/2015
• Issued: 03/15/2016
• Est. Priority Date: 01/12/2010
• Status: Active Grant

• Current Assignee: Sony Corporation (Sony Group Corp.)
• Original Assignee: Sony Corporation (Sony Group Corp.)
• Inventors: Fukuchi, Masaki, Matsuda, Kouichi, Suto, Yasuhiro, Oi, Kenichiro, Guo, Jingjing
• Primary Examiner(s): AZARIAN, SEYED H
• Application Number:
  US14/657,431
• Publication Number:
  US 20150189216A1
• Time in Patent Office: 368 Days
• Field of Search: 382/100, 382/103, 382106-107, 382/154, 382/162, 382/168, 382/173, 382/181, 382/189, 382/206, 382/214, 382/224, 382/232, 382/254, 382/276, 382286-291, 382/305, 382/312, 345/156, 345/158, 345/169, 404/31, 701/24, 348/734, 718/100
• US Class Current: 1/1
• CPC Class Codes: G06F 3/005 Input arrangements through a video camera
• CPC Class Codes: G08C 2201/20 Binding and programming of remote control devices
• CPC Class Codes: H04N 21/4108 characterised by an identification number or address, e.g. local network address protecting specific internal or external computer components using identification number G06F21/73; network arrangements, protocols or services for addressing or naming H04L61/00
• CPC Class Codes: H04N 21/42204 User interfaces specially adapted for controlling a client device through a remote control device; Remote control devices therefor constructive details of casings for the remote control device H01H9/0235; user interfaces for controlling a tuning device of a television receiver through a remote control H03J9/00; remote control of peripheral devices connected to a television receiver through the remote control device of the television receiver H04B1/205
• CPC Class Codes: H04N 21/42208 Display device provided on the remote control
• CPC Class Codes: H04N 21/42209 for displaying non-command information, e.g. electronic program guide [EPG], e-mail, messages or a second television channel
• CPC Class Codes: H04N 21/4222 Remote control device emulator integrated into a non-television apparatus, e.g. a PDA, media center or smart toy remote control device for a television receiver integrated into a mobile phone H04M1/72415
• CPC Class Codes: H04N 21/42222 Additional components integrated in the remote control device, e.g. timer, speaker, sensors for detecting position, direction or movement of the remote control, microphone or battery charging device
• CPC Class Codes: H04N 21/42226 Reprogrammable remote control devices
• CPC Class Codes: H04N 21/4223 Cameras H04N23/00 takes precedence
• CPC Class Codes: H04N 21/43615 Interfacing a Home Network, e.g. for connecting the client to a plurality of peripherals home Audio Video Interoperability [HAVI] data switching networks H04L12/2805
• CPC Class Codes: H04N 21/44008 involving operations for analysing video streams, e.g. detecting features or characteristics in the video stream arrangements characterised by components specially adapted for monitoring, identification or recognition of video in broadcast systems H04H60/59

Abstract

There is provided an image processing device including: a data storage unit that stores object identification data for identifying an object operable by a user and feature data indicating a feature of appearance of each object; an environment map storage unit that stores an environment map representing a position of one or more objects existing in a real space and generated based on an input image obtained by imaging the real space using an imaging device and the feature data stored in the data storage unit; and a selecting unit that selects at least one object recognized as being operable based on the object identification data, out of the objects included in the environment map stored in the environment map storage unit, as a candidate object being a possible operation target by a user.

Claims

1. An image processing apparatus comprising:

a circuitry configured to control a display device to display a command information to be perceived by a user as located at a position within a real space or a representation of the real space based on three-dimensional positioning information of the real space,

wherein the command information is controlled to be displayed at a fixed relative display location relative to an object that is present either within the real space or within the representation of the real space,

wherein the command information indicates a command for operating the object,

wherein the command information is displayed based on a position and attitude of the display device, and

wherein the command information is displayed to be superimposed upon an image containing the object.

2. The image processing apparatus according to claim 1, wherein the three-dimensional positioning information indicates at least a coordinate location of the object.

3. The image processing apparatus according to claim 1, wherein the three-dimensional positioning information of the real space provides for a mapping model of the real space.

4. The image processing apparatus according to claim 1, wherein the command information specifies how a user should operate the object.

5. The image processing apparatus according to claim 1, wherein the command information indicates the command for the user to operate the object.

6. The image processing apparatus according to claim 1, wherein the command information indicates that the object is an operable object.

7. The image processing apparatus according to claim 1, wherein the command information indicates a timing at which the user should operate the object.

8. The image processing apparatus according to claim 1, wherein the command information indicates a name of the object.

9. The image processing apparatus according to claim 1, wherein the command information indicates a list of commands for operating the object.

10. The image processing apparatus according to claim 1, wherein the command information indicates candidate operations of the object.

11. The image processing apparatus according to claim 1, wherein the command information is displayed based on an image detection of the object.

12. The image processing apparatus according to claim 1, wherein the circuitry controls the display device to display the command information to be perceived by the user as located at a position within the real space.

13. The image processing apparatus according to claim 1, wherein the circuitry controls the display device to display the command information to be perceived by the user as located at a position within an imaged representation of the real space.

14. The image processing apparatus according to claim 1, wherein the three-dimensional positional information of the real space defines an environment map.

15. The image processing apparatus according to claim 1, wherein the command information comprises an identification of the object.

16. An image processing method comprising:

controlling a display device to display a command information to be perceived by a user as located at a position within a real space or a representation of the real space based on three-dimensional positioning information of the real space,

wherein the command information is controlled to be displayed at a fixed relative display location relative to an object that is present either within the real space or within the representation of the real space,

wherein the command information indicates a command for operating the object,

wherein the command information is displayed based on a position and attitude of the display device, and

wherein the command information is displayed to be superimposed upon an image containing the object.

17. The image processing method according to claim 16, wherein the three-dimensional positional information of the real space defines an environment map.

18. A non-transitory computer-readable medium having embodied thereon a program, which when executed by a computer causes the computer to execute an image processing method, the method comprising:

controlling a display device to display a command information to be perceived by a user as located at a position within a real space or a representation of the real space based on three-dimensional positioning information of the real space,

wherein the command information is controlled to be displayed at a fixed relative display location relative to an object that is present either within the real space or within the representation of the real space,

wherein the command information indicates a command for operating the object,

wherein the command information is displayed based on a position and attitude of the display device, and

wherein the command information is displayed to be superimposed upon an image containing the object.

19. The non-transitory computer-readable medium according to claim 18, wherein the three-dimensional positional information of the real space defines an environment map.