SYSTEMS, DEVICES, AND METHODS FOR WEARABLE HEADS-UP DISPLAYS AS WIRELESS CONTROLLERS

• US 20170153701A1
• Filed: 11/29/2016
• Published: 06/01/2017
• Est. Priority Date: 12/01/2015
• Status: Abandoned Application

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Thalmic Labs Inc.
• Inventors: Mahon, Thomas, Bisaillion, Brent
• Application Number:
  US15/363,970
• Publication Number:
  US 20170153701A1
• Time in Patent Office: Days
• CPC Class Codes: G02B 2027/0187 slaved to motion of at least a part of the body of the user, e.g. head, eye
• CPC Class Codes: G02B 27/017 Head mounted
• CPC Class Codes: G06F 2203/0384 Wireless input, i.e. hardware and software details of wireless interface arrangements for pointing devices
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: G06F 3/0383 Signal control means within the pointing device
• CPC Class Codes: G06F 3/04817 using icons graphical or visual programming using iconic symbols G06F8/34
• CPC Class Codes: G06F 3/0482 Interaction with lists of selectable items, e.g. menus
• CPC Class Codes: G06F 3/0484 for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range
• CPC Class Codes: G06F 3/04842 Selection of displayed objects or displayed text elements G06F3/0482 takes precedence
• CPC Class Codes: G06F 3/04847 Interaction techniques to control parameter settings, e.g. interaction with sliders or dials
• CPC Class Codes: H04N 21/42204 User interfaces specially adapted for controlling a client device through a remote control device; Remote control devices therefor constructive details of casings for the remote control device H01H9/0235; user interfaces for controlling a tuning device of a television receiver through a remote control H03J9/00; remote control of peripheral devices connected to a television receiver through the remote control device of the television receiver H04B1/205
• CPC Class Codes: H04N 21/42208 Display device provided on the remote control
• CPC Class Codes: H04N 21/4222 Remote control device emulator integrated into a non-television apparatus, e.g. a PDA, media center or smart toy remote control device for a television receiver integrated into a mobile phone H04M1/72415
• CPC Class Codes: H04N 21/42222 Additional components integrated in the remote control device, e.g. timer, speaker, sensors for detecting position, direction or movement of the remote control, microphone or battery charging device
• CPC Class Codes: H04N 21/44218 Detecting physical presence or behaviour of the user, e.g. using sensors to detect if the user is leaving the room or changes his face expression during a TV program methods or arrangements for recognising human body or animal bodies or body parts G06V40/10; methods or arrangements for acquiring or recognising human faces, facial parts, facial sketches, facial expressions G06V40/16; methods or arrangements for recognising movements or behaviour G06V40/20; arrangements for identifying users in broadcast systems H04H60/45

Abstract

Systems, devices, and methods that operate a wearable heads-up display (“WHUD”) as a remote controller to wirelessly control at least one other electronic device are described. The WHUD displays a visual control interface including a set of user-selectable icons that each correspond to a respective function for the electronic device under wireless/remote control. The WHUD includes an eye tracker that detects when the user is looking/gazing at a particular one of the icons in the visual control interface. The user provides an indication that he/she wishes to select the particular icon at which he/she is gazing/looking (e.g., by dwelling his/her gaze on the particular icon or by performing a selection action via a separate portable interface device). In response, the WHUD wirelessly transmits a signal that provides data and/or instructions for the electronic device under wireless/remote control to effect the particular function selected by the user.

Claims

1. A method of operating a wearable system to wirelessly control an electronic device, the method comprising:

displaying, by a wearable heads-up display, a visual control interface for the electronic device, the visual control interface including a set of user-selectable icons that each correspond to a respective function for the electronic device;

detecting, by an eye tracker of the wearable heads-up display, that a user of the wearable heads-up display is gazing at a particular user-selectable icon in the visual control interface;

receiving, by the wearable heads-up display, an indication from the user to select the particular user-selectable icon in the visual control interface; and

wirelessly transmitting, by a wireless transmitter of the wearable heads-up display, a wireless signal to effect a function of the electronic device corresponding to the particular user-selectable icon selected by the user.

2. The method of claim 1 wherein receiving, by the wearable heads-up display, an indication from the user to select the particular user-selectable icon in the visual control interface includes detecting, by the eye tracker of the wearable heads-up display, that the user is continuously gazing at the particular user-selectable icon for a defined amount of time.

3. The method of claim 2 wherein the defined amount of time is selected from a group consisting of:

about one second, about two seconds, about three seconds, about four seconds, and about five seconds.

4. The method of claim 1 wherein receiving, by the wearable heads-up display, an indication from the user to select the particular user-selectable icon in the visual control interface includes receiving, by a wireless receiver of the wearable heads-up display, a wireless signal transmitted from a portable interface device, the wireless signal representative of a deliberate selection action performed by the user while the eye tracker of the wearable heads-up display is detecting that the user is gazing at the particular user-selectable icon in the visual control interface.

5. The method of claim 4 wherein the portable interface device is selected from a group consisting of:

a smartphone, a gesture control armband, a wearable device, a portable interface device, and a batteryless and wireless portable interface device.

6. The method of claim 1, further comprising:

receiving, by the electronic device, the wireless signal wirelessly transmitted by the wearable heads-up display; and

effecting, by the electronic device, a function of the electronic device corresponding to the particular user-selectable icon selected by the user.

7. The method of claim 1 wherein the electronic device is selected from a group consisting of:

a remote-controlled device, a television, a personal computer, a laptop computer, a music player, a telephone, a thermostat, a light bulb, a radio, and a video game console.

8. The method of claim 1 wherein the wearable heads-up display includes a processor and a non-transitory processor-readable storage medium communicatively coupled to the processor, and wherein the non-transitory processor-readable storage medium stores data and/or processor-executable instructions, and wherein:

displaying, by a wearable heads-up display, a visual control interface for the electronic device includes executing, by the processor, the data and/or processor-executable instructions to cause the wearable heads-up display to display the visual control interface for the electronic device;

detecting, by an eye tracker of the wearable heads-up display, that a user of the wearable heads-up display is gazing at a particular user-selectable icon in the visual control interface includes executing, by the processor, the data and/or processor-executable instructions to cause the eye tracker of the wearable heads-up display to detect that the user is gazing at the particular user-selectable icon in the visual control interface; and

wirelessly transmitting, by a wireless transmitter of the wearable heads-up display, a wireless signal to effect a function of the electronic device corresponding to the particular user-selectable icon selected by the user includes executing, by the processor, the data and/or processor-executable instructions to cause the wireless transmitter of the wearable heads-up display to wirelessly transmit the wireless signal to effect the function of the electronic device corresponding to the particular user-selectable icon selected by the user.

9. The method of claim 1 wherein the set of user-selectable icons in the visual control interface displayed by the wearable heads-up display includes at least one user-selectable icon selected from a group consisting of:

a graphical icon corresponding to a particular function for the electronic device, a textual icon corresponding to a particular function for the electronic device, a pictorial icon corresponding to a particular function for the electronic device, and a combined textual and pictorial icon corresponding to a particular function for the electronic device.

10. The method of claim 1 wherein receiving, by the wearable heads-up display, an indication from the user to select the particular user-selectable icon in the visual control interface includes detecting, by the eye tracker of the wearable heads-up display, that the user is gazing at a selection button in the visual control interface after detecting, by the eye tracker of the wearable heads-up display, that the user is gazing at a particular user-selectable icon in the visual control interface.

11. A wearable system operative to wirelessly control an electronic device, the wearable system comprising:

a wearable heads-up display that includes;

a processor;

an eye tracker communicatively coupled to the processor;

a wireless transmitter communicatively coupled to the processor; and

a non-transitory processor-readable storage medium communicatively coupled to the processor, wherein the non-transitory processor-readable storage medium stores data and/or processor-executable instructions that, when executed by the processor, cause;

the wearable heads-up display to display a visual control interface for the electronic device, the visual control interface including a set of user-selectable icons that each correspond to a respective function for the electronic device;

the eye tracker to detect that a user of the wearable heads-up display is gazing at a particular user-selectable icon in the visual control interface; and

in response to receiving, by the wearable heads-up display, an indication from the user to select the particular user-selectable icon in the visual control interface, the wireless transmitter of the wearable heads-up display to wirelessly transmit a wireless signal to effect a function of the electronic device corresponding to the particular user-selectable icon selected by the user.

12. The wearable system of claim 11 wherein the electronic device is selected from a group consisting of:

a remote-controlled device, a television, a personal computer, a laptop computer, a music player, a telephone, a thermostat, a light bulb, a radio, and a video game console.

13. The wearable system of claim 11 wherein the set of user-selectable icons in the visual control interface displayed by the wearable heads-up display includes at least one user-selectable icon selected from a group consisting of:

a graphical icon corresponding to a particular function for the electronic device, a textual icon corresponding to a particular function for the electronic device, a pictorial icon corresponding to a particular function for the electronic device, and a combined textual and pictorial icon corresponding to a particular function for the electronic device.

14. The wearable system of claim 11 wherein the data and/or processor-executable instructions, when executed by the processor, further cause the eye tracker to detect that the user is continuously gazing at the particular user-selectable icon for a defined amount of time and, in response to detecting that the user is continuously gazing at the particular user-selectable icon for the defined amount of time, provide the indication to select the particular user-selectable icon in the visual control interface.

15. The wearable system of claim 14 wherein the defined amount of time is selected from a group consisting of:

about one second, about two seconds, about three seconds, about four seconds, and about five seconds.

16. The wearable system of claim 11, further comprising a portable interface device that in use is carried or worn by the user, wherein the portable interface device includes at least one actuator that, when activated by the user, causes the portable interface device to wirelessly transmit a selection signal, and wherein:

the data and/or processor-executable instructions stored in the non-transitory processor-readable storage medium of the wearable heads-up display that, when executed by the processor of the wearable heads-up display, cause, in response to receiving, by the wearable heads-up display, an indication from the user to select the particular user-selectable icon in the visual control interface, the wireless transmitter of the wearable heads-up display to wirelessly transmit a wireless signal to effect a function of the electronic device corresponding to the particular user-selectable icon selected by the user, cause;

in response to wirelessly receiving, by the wearable heads-up display, the selection signal from the portable interface device, the wireless transmitter of the wearable heads-up display to wirelessly transmit a wireless signal to effect a function of the electronic device corresponding to the particular user-selectable icon selected by the user.

17. The wearable system of claim 16 wherein the portable interface device is selected from a group consisting of:

a smartphone, a gesture control armband, a wearable device, a portable interface device, and a batteryless and wireless portable interface device.

18. The wearable system of claim 11, further comprising a portable interface device that in use is carried or worn by the user, wherein the portable interface device includes at least one actuator that, when activated by the user, causes the portable interface device to wirelessly transmit a selection signal, and wherein the indication from the user to select the particular user-selectable icon in the visual control interface includes a receipt, by the wireless receiver of the wearable heads-up display, of the selection signal wirelessly transmitted by the portable interface device when the at least one actuator of the portable interface device is activated by the user.

19. The wearable system of claim 11 wherein the data and/or processor-executable instructions, when executed by the processor, further cause the eye tracker to detect that the user is gazing at a selection button in the visual control interface after detecting that the user is gazing at a particular user-selectable icon in the visual control interface and, in response to detecting that the user is gazing at the selection button in the visual control interface after detecting that the user is gazing at the particular user-selectable icon in the visual control interface, provide the indication to select the particular user-selectable icon in the visual control interface.