APPARATUS FOR CONSTRUCTING DEVICE INFORMATION FOR CONTROL OF SMART APPLIANCES AND METHOD THEREOF

• US 20140257532A1
• Filed: 05/31/2013
• Published: 09/11/2014
• Est. Priority Date: 03/05/2013
• Status: Abandoned Application

• Current Assignee: Electronics and Telecommunications Research Institute
• Original Assignee: Electronics and Telecommunications Research Institute
• Inventors: KIM, Sung-Soo, CHO, Chunglae, SON, Ji Yeon, PARK, Jun Hee, PARK, Kwang Roh
• Application Number:
  US13/907,725
• Publication Number:
  US 20140257532A1
• Time in Patent Office: Days
• US Class Current: 700/90
• CPC Class Codes: G05B 15/02 electric
• CPC Class Codes: G05B 19/0423 Input/output
• CPC Class Codes: G05B 2219/2642 Domotique, domestic, home control, automation, smart house
• CPC Class Codes: G05B 2219/35444 Gesture interface, controlled machine observes operator, executes commands
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0304 Detection arrangements using opto-electronic means constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042

Abstract

An apparatus for constructing device information for control of smart appliances includes a camera member, which generates multi-view images for 3D reconstruction through multi-angle photographing in an indoor space using a depth camera, and generates 3D data using the generated multi-view images, and a home device controller, which constructs a 3D space using the generated 3D data, and constructs a device profile for controlling an operation of a device to be controlled, which is designated by a user, through combination of spatial data of the device to be controlled and the constructed 3D space.

Claims

1. An apparatus for constructing device information for control of smart appliances, comprising:

a camera member, which generates multi-view images for 3D reconstruction through multi-angle photographing in an indoor space using a depth camera, and generates 3D data using the generated multi-view images; and

a home device controller, which constructs a 3D space using the generated 3D data, and constructs a device profile for controlling an operation of a device to be controlled, which is designated by a user, through combination of spatial data of the device to be controlled and the constructed 3D space.

2. The apparatus for constructing device information of claim 1, wherein the camera member comprises:

a depth camera scan unit, which generates the images through the multi-angle photographing and capturing that scan the indoor space in upward, downward, left, and right directions;

a multi-view image processing unit, which processes the images generated for construction of a 3D scene from the multi-view images; and

a 3D data generation unit, which generates the 3D data through reconstruction of the processed multi-view images.

3. The apparatus for constructing device information of claim 2, further comprising a candidate region extraction unit, which extracts a candidate region of the device to be controlled in the 3D space that is constructed using the generated 3D data.

4. The apparatus for constructing device information of claim 2, wherein the multi-view image processing unit processes the multi-view images using an iterative closest point (ICP) algorithm.

5. The apparatus for constructing device information of claim 2, wherein the multi-view image processing unit processes the multi-view images using a random sample consensus (RANSAC) algorithm.

6. The apparatus for constructing device information of claim 1, wherein the multi-view image includes RGB information for representing colors by pixels that constitute the image and depth information.

7. The apparatus for constructing device information of claim 1, wherein the user designation is a designation based on user gestures.

8. The apparatus for constructing device information of claim 1, wherein the home device controller comprises:

a 3D data construction unit, which constructs the 3D space using the generated 3D data;

a device search unit, which searches for the device to be controlled that exists in a home network and generates and provides a device list; and

a device information combination unit, which constructs the device profile for controlling the operation of the device to be controlled through combination of device attribute information of the designated device to be controlled and information about a bounding region in the 3D space when any one of the devices to be controlled in the device list is designated.

9. The apparatus for constructing device information of claim 8, wherein the device attribute information includes 3D bounding box information, basic device information, detailed device information, device control GUI information, and device status information.

10. A method for constructing device information for control of smart appliances, comprising:

generating multi-view images for 3D reconstruction through multi-angle photographing in an indoor space using a depth camera, and then generating 3D data using the generated multi-view images; and

constructing a 3D space using the generated 3D data to express the 3D space on a display panel of a user terminal, and then constructing a device profile for controlling an operation of a device to be controlled through combination of attribute information of the device to be controlled and the 3D space.

11. A method for constructing device information for control of smart appliances, comprising:

generating multi-view images for 3D reconstruction through multi-angle photographing by a depth camera connected to a home network;

generating 3D data using the generated multi-view images and displaying the 3D data on a display panel of a user terminal;

determining a bounding region for a device to be controlled in a 3D space constructed through the 3D data;

generating a device list through searching for the device to be controlled that exists in the home network and expressing the generated device list on the display panel; and

constructing a device profile for controlling operation of the device to be controlled through combination of device attribute information of the designated device to be controlled and information of the bounding region when any one of the devices to be controlled in the generated device list is designated.

12. The method for constructing device information of claim 11, wherein the multi-view image is acquired through scan photographing and capturing in upward, downward, left, and right directions in an indoor space where the depth camera is installed.

13. The method for constructing device information of claim 12, wherein the multi-view image includes RGB information for representing colors by pixels that constitute the image and depth information.

14. The method for constructing device information of claim 11, wherein the 3D data is generated through application of an iterative closest point (ICP) algorithm to the multi-view images.

15. The method for constructing device information of claim 11, wherein the 3D data is generated through application of a random sample consensus (RANSAC) algorithm to the multi-view images.

16. The method for constructing device information of claim 11, wherein the designation of the device to be controlled is a designation based on user gestures.

17. The method for constructing device information of claim 11, wherein the device attribute information includes 3D bounding box information, basic device information, detailed device information, device control GUI information, and device status information.

18. The method for constructing device information of claim 11, further comprising constructing a new device profile for controlling the operation of the device through proceeding with a related process whenever a new device to be controlled among the devices to be controlled in the generated device list is designated.

19. The method for constructing device information of claim 11, further comprising storing the constructed device profile for controlling the operation of the device to be controlled in a profile DB.

20. The method for constructing device information of claim 11, wherein the constructing the device profile comprises:

acquiring box information of a 3D bounding region based on user designation;

acquiring detailed device information about the device to be controlled that coincides with the corresponding bounding region;

acquiring device control GUI information for controlling the corresponding device to be controlled; and

generating the device profile for controlling the operation of the device to be controlled through combination of the acquired detailed device information and device control GUI information and the box information of the bounding region as the device attribute information.