Electronic device and method for controlling external object

• US 10,055,015 B2
• Filed: 10/30/2015
• Issued: 08/21/2018
• Est. Priority Date: 11/03/2014
• Status: Active Grant

• Current Assignee: Samsung Electronics Co. Ltd.
• Original Assignee: Samsung Electronics Co. Ltd.
• Inventors: Hwang, Seong-Hee
• Primary Examiner(s): Amadiz, Rodney
• Application Number:
  US14/928,338
• Publication Number:
  US 20160124506A1
• Time in Patent Office: 1,026 Days
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06T 1/20 Processor architectures; Processor configuration, e.g. pipelining

Abstract

Methods and apparatuses are provided for controlling an external object by an electronic device. A line of sight of a user is determined using an image sensor of the electronic device. An object located outside of the electronic device is determined based on the line of sight of the user. Object information regarding the object is determined. A user input with respect to the object is received from the user. The object or another electronic device associated with the object is controlled based on the user input and the object information.

Claims

1. A method for controlling a target object by a first electronic device, the method comprising:

determining a line of sight of a user by synchronizing coordinates between an eye image of the user captured using a first camera of the first electronic device and a view image generated using a second camera of the first electronic device;

determining a target object located outside of the electronic device based on the line of sight;

determining whether the target object is capable of communicating with the first electronic device;

if the target object is not a machine and is not capable of communicating with the first electronic device, generating a control signal for controlling a second electronic device capable of communicating with the first electronic device, wherein the second electronic device is capable of changing a state of the target object; and

transmitting the control signal to the second electronic device such that the second electronic device changes the state of the target object based on the control signal.

2. The method of claim 1, further comprising:

determining a direction of the line of sight of a user or a size of a pupil.

3. The method of claim 1, wherein determining the target object comprises:

determining an image region of the target object based on a contour of at least a part of an image associated with the target object; and

determining object information about the target object based on the image region.

4. The method of claim 3, wherein determining the image region comprises:

determining the image region according to a gesture input or a voice input, which is inputted through the first electronic device.

5. The method of claim 3, wherein determining the object information about the target information comprises:

determining, as the object information, at least one of a brand name associated with the target object, a form of the target object, an arrangement form of a button included in the target object, a color of the target object, an accessory attached to the target object, a trademark of the target object, and text information associated with the target object.

6. The method of claim 1, wherein generating the control signal for controlling the second electronic device comprises:

determining device information about the second electronic device;

comparing the device information with a device list pre-stored or a control history pre-stored; and

generating the control signal for controlling the second electronic device based on the comparison result.

7. The method of claim 1, wherein generating the control signal for controlling the second electronic device comprises:

receiving control information for controlling the second electronic device from an external server; and

generating the control signal for controlling the second electronic device based on the control information.

8. The method of claim 1, wherein generating the control signal for controlling the second electronic device comprises:

displaying one or more items for controlling the second electronic device on at least an area of a display device functionally connected to the first electronic device; and

if an item from among the one or more items is selected, generating the control signal for controlling the second electronic device based on the selected item.

9. A first electronic device for controlling a target object, the electronic device comprising:

a first camera and a second camera;

a transceiver;

a controller configured to;

determine a line of sight of a user by synchronizing coordinates between an eye image of the user captured using the first camera and a view image generated using the second camera,

determine a target object located outside of the first electronic device based on the line of sight,

determine whether the target object is capable of communicating with the first electronic device, and

if the target object is not a machine and is not capable of communicating with the first electronic device, generate a control signal for controlling a second electronic device capable of communicating with the first electronic device, wherein the second electronic device is capable of changing a state of the target object; and

control the transceiver to transmit the control signal to the second electronic device such that the second electronic device changes the state of the target object based on the control signal.

10. The first electronic device of claim 9, wherein the controller is further configured to determine a direction of the line of sight of a user or a size of a pupil.

11. The first electronic device of claim 9, wherein the controller is further configured to:

determine an image region of the target object based on a contour of at least a part of an image associated with the target object, and

determine object information about the target object based on the image region.

12. The first electronic device of claim 11, wherein the controller is further configured to:

determine the image region according to a gesture input or a voice input, which is inputted through the electronic device.

13. The first electronic device of claim 11, wherein the controller is further configured to:

determine, as the object information, at least one of a brand name associated with the target object, a form of the target object, an arrangement form of a button included in the target object, a color of the target object, an accessory attached to the target object, a trademark of the target object, and text information associated with the target object.

14. The first electronic device of claim 11, wherein the controller is further configured to:

determine device information about the second electronic device,

compare the device information with a device list pre-stored or a control history pre-stored, and

generate the control signal for controlling the second electronic device based on the comparison result.

15. The first electronic device of claim 9, wherein the controller is further configured to:

control the transceiver to receive control information for controlling the second electronic device from an external server, and

generate the control signal for controlling the second electronic device based on the control information.

16. The first electronic device of claim 9, wherein the controller is further configured to:

control a display device functionally connected to the electronic device to display information for controlling one or more items for controlling the second electronic device, and

if a item from among the one or more items is selected, generate the control signal for controlling the second electronic device based the selected item selected.