IMAGE PROCESSING DEVICE, OBJECT SELECTION METHOD AND PROGRAM

• US 20140003667A1
• Filed: 08/30/2013
• Published: 01/02/2014
• Est. Priority Date: 01/12/2010
• Status: Active Grant

• Current Assignee: Sony Corporation (Sony Group Corp.)
• Original Assignee: Sony Corporation (Sony Group Corp.)
• Inventors: FUKUCHI, Masaki, MATSUDA, Kouichi, SUTO, Yasuhiro, OI, Kenichiro, GUO, Jingjing
• Granted Patent: US 8,744,130 B2
• Time in Patent Office: Days
• US Class Current: 382/103
• CPC Class Codes: G06F 3/005 Input arrangements through a video camera
• CPC Class Codes: G08C 2201/20 Binding and programming of remote control devices
• CPC Class Codes: H04N 21/4108 characterised by an identification number or address, e.g. local network address protecting specific internal or external computer components using identification number G06F21/73; network arrangements, protocols or services for addressing or naming H04L61/00
• CPC Class Codes: H04N 21/42204 User interfaces specially adapted for controlling a client device through a remote control device; Remote control devices therefor constructive details of casings for the remote control device H01H9/0235; user interfaces for controlling a tuning device of a television receiver through a remote control H03J9/00; remote control of peripheral devices connected to a television receiver through the remote control device of the television receiver H04B1/205
• CPC Class Codes: H04N 21/42208 Display device provided on the remote control
• CPC Class Codes: H04N 21/42209 for displaying non-command information, e.g. electronic program guide [EPG], e-mail, messages or a second television channel
• CPC Class Codes: H04N 21/4222 Remote control device emulator integrated into a non-television apparatus, e.g. a PDA, media center or smart toy remote control device for a television receiver integrated into a mobile phone H04M1/72415
• CPC Class Codes: H04N 21/42222 Additional components integrated in the remote control device, e.g. timer, speaker, sensors for detecting position, direction or movement of the remote control, microphone or battery charging device
• CPC Class Codes: H04N 21/42226 Reprogrammable remote control devices
• CPC Class Codes: H04N 21/4223 Cameras H04N23/00 takes precedence
• CPC Class Codes: H04N 21/43615 Interfacing a Home Network, e.g. for connecting the client to a plurality of peripherals home Audio Video Interoperability [HAVI] data switching networks H04L12/2805
• CPC Class Codes: H04N 21/44008 involving operations for analysing video streams, e.g. detecting features or characteristics in the video stream arrangements characterised by components specially adapted for monitoring, identification or recognition of video in broadcast systems H04H60/59

Abstract

There is provided an image processing device including: a data storage unit that stores object identification data for identifying an object operable by a user and feature data indicating a feature of appearance of each object; an environment map storage unit that stores an environment map representing a position of one or more objects existing in a real space and generated based on an input image obtained by imaging the real space using an imaging device and the feature data stored in the data storage unit; and a selecting unit that selects at least one object recognized as being operable based on the object identification data, out of the objects included in the environment map stored in the environment map storage unit, as a candidate object being a possible operation target by a user.

Claims

1. An image processing device comprising:

an imaging unit configured to output an image generated by imaging a real space;

a generating unit configured to generate an environment map representing a position of one or more objects existing in the real space based on the image and a feature data indicating a feature of appearance of each object; and

a selecting unit configured to select at least one operable object recognized as being operable based on the object identification data as a candidate object being a possible operation target by a user.

2. The image processing device according to claim 1, further comprising:

an image output unit configured to generate an output image including an indication of the position of the candidate object from the input image.

3. The image processing device according to claim 1, further comprising:

a device recognizing unit configured to recognize at least one operable object based on the image and the object identification data,

wherein the at least one operable object is recognized by the device recognizing unit.

4. The image processing device according to claim 1, further comprising:

a user interface for allowing the user to specify an object to be operated, out of candidate objects.

5. The image processing device according to claim 4, wherein

the user interface determines an object located at a center or near the center of the output image at a time when user input is detected, out of candidate objects, as the object to be operated.

6. The image processing device according to claim 5, wherein

when a plurality of candidate objects are located at the center of the output image when user input is detected, the user interface determines the candidate object with a smallest size in the output image, out of candidate objects, as the object to be operated.

7. The image processing device according to claim 1, wherein

the feature data includes shape data indicating a three-dimensional shape of each object, and

the image output unit generates the output image by generating a frame surrounding each candidate object on the screen based on the shape data and superposing the generated frame on the input image.

8. The image processing device according to claim 1, further comprising:

a display device;

wherein an imaging plane of the imaging unit and a screen of the display device are arranged to face opposite directions from each other.

11. The image processing device according to claim 3, wherein the device recognizing unit is further configured to calculate the position of the operable object and register the operable object in the environment map.

12. The image processing device according to claim 2, wherein the imaging unit and the image output unit are both equipped in the image processing device.

13. The image processing device according to claim 1, wherein

the selection unit is further configured to select all the operable objects based on the object identification data.

14. The image processing device according to claim 1, wherein

the selection unit is further configured to select the operable objects so as not to exceed a preset threshold.

15. The image processing device according to claim 4, further comprising:

a transmitting unit configured to generate a control signal containing a control command for the specified object and transmit the generated control signal, the control command being associated with the object identification data.

9. (canceled)

10. (canceled)

16. An image processing method comprising:

outputting an image generated by imaging a real space;

generating an environment map representing a position of one or more objects existing in the real space based on the image and a feature data indicating a feature of appearance of each object; and

selecting at least one operable object recognized as being operable based on the object identification data as a candidate object being a possible operation target by a user.

17. A non-transitory computer-readable medium having embodied thereon a program, which when executed by a computer causes the computer to perform a method, the method comprising:

outputting an image generated by imaging a real space;

generating an environment map representing a position of one or more objects existing in the real space based on the image and a feature data indicating a feature of appearance of each object; and

selecting at least one operable object recognized as being operable based on the object identification data as a candidate object being a possible operation target by a user.