User interface

• US 10,136,104 B1
• Filed: 12/21/2015
• Issued: 11/20/2018
• Est. Priority Date: 01/09/2012
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google LLC (Alphabet Inc.)
• Inventors: Spitzer, Mark B.
• Primary Examiner(s): Topgyal, Gelek W
• Application Number:
  US14/976,368
• Time in Patent Office: 1,065 Days
• Field of Search: 348 42- 60
• CPC Class Codes: G02B 2027/0138 comprising image capture systems, e.g. camera
• CPC Class Codes: G02B 2027/014 comprising information/image processing systems
• CPC Class Codes: G02B 2027/0178 Eyeglass type eyeglass details G02C
• CPC Class Codes: G02B 2027/0187 slaved to motion of at least a part of the body of the user, e.g. head, eye
• CPC Class Codes: G02B 27/0093 with means for monitoring data relating to the user, e.g. head-tracking, eye-tracking
• CPC Class Codes: G02B 27/017 Head mounted
• CPC Class Codes: G02B 27/0172 characterised by optical features
• CPC Class Codes: G02B 27/0179 Display position adjusting means not related to the information to be displayed
• CPC Class Codes: G06F 1/163 Wearable computers, e.g. on a belt
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: H04N 13/366 using viewer tracking
• CPC Class Codes: H04N 13/383 for tracking with gaze detection, i.e. detecting the lines of sight of the viewer's eyes
• CPC Class Codes: H04N 5/76 Television signal recording
• CPC Class Codes: H04N 7/18 Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast
• CPC Class Codes: H04N 7/181 for receiving images from a plurality of remote sources

Abstract

Methods and apparatus for controlling one or more devices using a computing device, such as a wearable computing device, are disclosed. Using a sensor associated with a wearable computing device, one or more inputs are generated at the wearable computing device. As one example, the sensor can be an image capture device, and the inputs can be one or more images from a point of view of a person associated with the computing device. As another example, the sensor can be a motion sensor, and the input can be locations and/or orientations, and a gaze direction can be determined. The computing device can determine an object of interest based on the inputs. A control signal can be generated for the object of interest. The control signal can be transmitted from the computing device.

Claims

1. A wearable computing device comprising:

one or more processors;

a user interface; and

a non-transitory computer readable medium storing instructions that, when executed by the one or more processors, cause the wearable computing device to perform functions comprising;

capturing two or more images corresponding respectively to two or more devices within an environment of the wearable computing device;

receiving, via the user interface, input indicating two or more device types corresponding respectively to the two or more captured images;

storing data indicating the correspondence between the two or more device types and the respective two or more captured images;

capturing an additional image of a device of the two or more devices from a point of view of a wearer of the wearable computing device;

using the stored data and the additional captured image to select a control signal configured to control the device; and

sending the selected control signal to the device.

2. The wearable computing device of claim 1, wherein using the stored data and the additional captured image to select the control signal comprises selecting the control signal based on determining that the device is represented in the additional captured image.

3. The wearable computing device of claim 2,

wherein capturing the additional image comprises capturing an additional two or more images, and

wherein determining that the device is represented in the additional captured image comprises determining that the device is represented in at least a threshold amount of images of the additional two or more images.

4. The wearable computing device of claim 2, wherein the control signal comprises a command for the device to perform at least one of:

powering on, playing media content, or resuming playback of media content.

5. The wearable computing device of claim 1, wherein using the stored data and the additional captured image to select the control signal comprises selecting the control signal based on determining that the device is not represented in the additional captured image.

6. The wearable computing device of claim 5,

wherein capturing the additional image comprises capturing an additional two or more images, and

wherein determining that the device is not represented in the additional captured image comprises determining that the device is not represented in at least a threshold amount of images of the additional two or more images.

7. The wearable computing device of claim 5, wherein the control signal comprises a command for the device to perform at least one of:

powering off or pausing playback of media content.

8. A method performed by a wearable computing device, the method comprising:

capturing two or more images corresponding respectively to two or more devices within an environment of the wearable computing device;

receiving, via a user interface, input indicating two or more device types corresponding respectively to the two or more captured images;

storing data indicating the correspondence between the two or more device types and the respective two or more captured images;

capturing an additional image of a device of the two or more devices from a point of view of a wearer of the wearable computing device;

using the stored data and the additional captured image to select a control signal configured to control the device; and

sending the selected control signal to the device.

9. The method of claim 8, wherein using the stored data and the additional captured image to select the control signal comprises selecting the control signal based on determining that the device is represented in the additional captured image.

10. The method of claim 9,

wherein capturing the additional image comprises capturing an additional two or more images, and

wherein determining that the device is represented in the additional captured image comprises determining that the device is represented in at least a threshold amount of images of the additional two or more images.

11. The method of claim 9, wherein the control signal comprises a command for the device to perform at least one of:

powering on, playing media content, or resuming playback of media content.

12. The method of claim 8, wherein using the stored data and the additional captured image to select the control signal comprises selecting the control signal based on determining that the device is not represented in the additional captured image.

13. The method of claim 12,

wherein capturing the additional image comprises capturing an additional two or more images, and

wherein determining that the device is not represented in the additional captured image comprises determining that the device is not represented in at least a threshold amount of images of the additional two or more images.

14. The method of claim 12, wherein the control signal comprises a command for the device to perform at least one of:

powering off or pausing playback of media content.

15. A non-transitory computer readable medium storing instructions that, when performed by a wearable computing device, cause the wearable computing device to perform functions comprising:

capturing two or more images corresponding respectively to two or more devices within an environment of the wearable computing device;

receiving, via a user interface, input indicating two or more device types corresponding respectively to the two or more captured images;

storing data indicating the correspondence between the two or more device types and the respective two or more captured images;

capturing an additional image of a device of the two or more devices from a point of view of a wearer of the wearable computing device;

using the stored data and the additional captured image to select a control signal configured to control the device; and

sending the selected control signal to the device.

16. The non-transitory computer readable medium of claim 15, wherein using the stored data and the additional captured image to select the control signal comprises selecting the control signal based on determining that the device is represented in the additional captured image.

17. The non-transitory computer readable medium of claim 16,

wherein capturing the additional image comprises capturing an additional two or more images, and

wherein determining that the device is represented in the additional captured image comprises determining that the device is represented in at least a threshold amount of images of the additional two or more images.

18. The non-transitory computer readable medium of claim 16, wherein the control signal comprises a command for the device to perform at least one of:

powering on, playing media content, or resuming playback of media content.

19. The non-transitory computer readable medium of claim 15, wherein using the stored data and the additional captured image to select the control signal comprises selecting the control signal based on determining that the device is not represented in the additional captured image.

20. The non-transitory computer readable medium of claim 19,

wherein capturing the additional image comprises capturing an additional two or more images, and

wherein determining that the device is not represented in the additional captured image comprises determining that the device is not represented in at least a threshold amount of images of the additional two or more images.