User interface

• US 9,256,071 B1
• Filed: 01/09/2012
• Issued: 02/09/2016
• Est. Priority Date: 01/09/2012
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Spitzer, Mark B.
• Primary Examiner(s): Topgyal, Gelek W
• Application Number:
  US13/346,352
• Time in Patent Office: 1,492 Days
• Field of Search: 348 42- 60, 348148-160, 345/8
• US Class Current: 1/1
• CPC Class Codes: G02B 2027/0138 comprising image capture systems, e.g. camera
• CPC Class Codes: G02B 2027/014 comprising information/image processing systems
• CPC Class Codes: G02B 2027/0178 Eyeglass type eyeglass details G02C
• CPC Class Codes: G02B 2027/0187 slaved to motion of at least a part of the body of the user, e.g. head, eye
• CPC Class Codes: G02B 27/0093 with means for monitoring data relating to the user, e.g. head-tracking, eye-tracking
• CPC Class Codes: G02B 27/017 Head mounted
• CPC Class Codes: G02B 27/0172 characterised by optical features
• CPC Class Codes: G02B 27/0179 Display position adjusting means not related to the information to be displayed
• CPC Class Codes: G06F 1/163 Wearable computers, e.g. on a belt
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: H04N 13/366 using viewer tracking
• CPC Class Codes: H04N 13/383 for tracking with gaze detection, i.e. detecting the lines of sight of the viewer's eyes
• CPC Class Codes: H04N 5/76 Television signal recording
• CPC Class Codes: H04N 7/18 Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast
• CPC Class Codes: H04N 7/181 for receiving images from a plurality of remote sources

Abstract

Methods and apparatus for controlling one or more devices using a computing device, such as a wearable computing device, are disclosed. Using a sensor associated with a wearable computing device, one or more inputs are generated at the wearable computing device. As one example, the sensor can be an image capture device, and the inputs can be one or more images from a point of view of a person associated with the computing device. As another example, the sensor can be a motion sensor, and the input can be locations and/or orientations, and a gaze direction can be determined. The computing device can determine an object of interest based on the inputs. A control signal can be generated for the object of interest. The control signal can be transmitted from the computing device.

Claims

1. A method, comprising:

using an image capture device associated with a wearable computing device, capturing a first image from a point of view of a wearer of the wearable computing device;

using the wearable computing device, determining a device of interest based on the first image;

using the wearable computing device, generating a first control signal for the device of interest to (i) begin playback of media content, (ii) resume playback of media content, or (iii) power on;

capturing a second image from the point of view of the wearer;

determining that the device of interest is not in the second image; and

in response to determining that the device of interest is not in the second image, generating a second control signal for the device of interest to (i) pause playback of media content or (ii) power off.

2. The method of claim 1, further comprising transmitting the first control signal and the second control signal from the wearable computing device.

3. The method of claim 1, wherein the first control signal comprises a signal for the device of interest to power on.

4. The method of claim 1, wherein determining the device of interest comprises:

recognizing the device of interest in the first image.

5. The method of claim 1, wherein the second control signal comprises a signal for the device of interest to power off.

6. The method of claim 1, wherein the first control signal comprises a signal for the device of interest to begin playback of media content.

7. The method of claim 1,

further comprising capturing a plurality of images over a period of time, wherein the plurality of images includes the second image,

wherein the second control signal is generated in response to determining that the device of interest is not in any image of the plurality of images.

8. The method of claim 1, further comprising:

capturing a plurality of images over a period of time;

determining that the device of interest is in at least one image of the plurality of images; and

based on determining that the device of interest is in at least one image of the plurality of images, generating a third control signal for the device of interest.

9. The method of claim 8, wherein the third control signal comprises a volume-control signal, an input-source-control signal, or a combination volume-and-input-source-control signal.

10. The method of claim 1, wherein the device of interest comprises:

a display device; and

a control device, wherein the control device is configured to receive the first and second control signals and control the display device based on receiving the first and second control signals.

11. A wearable computing device, comprising:

an image capture device;

a processor; and

memory having instructions that, in response to execution by the processor, cause the wearable computing device to perform functions comprising;

the image capture device capturing a first image from a point of view of a wearer of the wearable computing device;

determining a device of interest based on the first image;

generating a first control signal for the device of interest to (i) begin playback of media content, (ii) resume playback of media content, or (iii) power on;

capturing a second image from the point of view of the wearer;

determining that the device of interest is not in the second image; and

in response to determining that the device of interest is not in the second image, generating a second control signal for the device of interest to (i) pause playback of media content or (ii) power off.

12. The wearable computing device of claim 11, further comprising a command interface, wherein the functions further comprise transmitting the first control signal and the second control signal using the command interface.

13. The wearable computing device of claim 11, wherein the first control signal comprises a signal for the device of interest to power on.

14. The wearable computing device of claim 11, wherein determining the device of interest comprises:

recognizing the device of interest in the first image.

15. The wearable computing device of claim 11, wherein the second control signal comprises a signal for the device of interest to power off.

16. The wearable computing device of claim 11, wherein the first control signal comprises a signal for the device of interest to begin playback of media content.

17. The wearable computing device of claim 11,

wherein the functions further comprise capturing a plurality of images over a period of time, wherein the plurality of images includes the second image, and

wherein the second control signal is generated in response to determining that the device of interest is not in any image of the plurality of images.

18. The wearable computing device of claim 11, wherein the functions further comprise:

capturing a plurality of images over a period of time;

determining that the device of interest is in at least one image of the plurality of images; and

based on determining that the device of interest is in at least one image of the plurality of images, generating a third control signal for the device of interest.

19. The wearable computing device of claim 18, wherein the third control signal comprises a volume-control signal, an input-source-control signal, or a combination volume-and-input-source-control signal.

20. The wearable computing device of claim 11, wherein the device of interest comprises:

a display device; and

a control device, wherein the control device is configured to receive the first and second control signals and control the display device based on receiving the first and second control signals.

21. A non-transitory computer-readable storage medium having instructions stored thereon that, in response to execution by a wearable computing device, cause the wearable computing device to perform functions, the instructions comprising:

instructions for capturing a first image from a point of view of a wearer of the wearable computing device;

instructions for determining a device of interest based on the first image;

instructions for generating a first control signal for the device of interest to (i) begin playback of media content, (ii) resume playback of media content, or (iii) power on;

instructions for capturing a second image from the point of view of the wearer;

instructions for determining that the device of interest is not in the second image; and

instructions for, in response to determining that the device of interest is not in the second image, generating a second control signal for the device of interest to (i) pause playback of media content or (ii) power off.

22. The non-transitory computer-readable storage medium of claim 21, the instructions further comprising instructions for transmitting the first control signal and the second control signal.

23. The non-transitory computer-readable storage medium of claim 21, wherein the first control signal comprises a signal for the device of interest to power on.

24. The non-transitory computer-readable storage medium of claim 21, wherein the instructions for determining the device of interest comprise:

instructions for recognizing the device of interest in the first image.

25. The non-transitory computer-readable storage medium of claim 21, wherein the second control signal comprises a signal for the device of interest to power off.

26. The non-transitory computer-readable storage medium of claim 21, wherein the first control signal comprises a signal for the device of interest to begin playback of media content.

27. The non-transitory computer-readable storage medium of claim 21,

wherein the instructions further comprise instructions for capturing a plurality of images over a period of time, wherein the plurality of images includes the second image, and

wherein the second control signal is generated in response to determining that the device of interest is not in any image of the plurality of images.

28. The non-transitory computer-readable storage medium of claim 21, wherein the instructions further comprise:

instructions for capturing a plurality of images over a period of time,

instructions for determining that the device of interest is in at least one image of the plurality of images; and

instructions for, based on determining that the device of interest is in at least one image of the plurality of images, generating a third control signal for the device of interest.

29. The non-transitory computer-readable storage medium of claim 28, wherein the third control signal comprises a volume-control signal, an input-source-control signal, or a combination volume-and-input-source-control signal.

30. The non-transitory computer-readable storage medium of claim 21, wherein the device of interest comprises:

a display device; and

a control device, wherein the control device is configured to receive the first and second control signals and control the display device based on receiving the first and second control signals.