Remote control of host application using motion and voice commands

• US 9,235,262 B2
• Filed: 05/05/2010
• Issued: 01/12/2016
• Est. Priority Date: 05/08/2009
• Status: Active Grant

• Current Assignee: Kopin Corporation
• Original Assignee: Kopin Corporation
• Inventors: Jacobsen, Jeffrey J., Parkinson, Christopher, Pombo, Stephen A.
• Primary Examiner(s): Bilodeau, David
• Application Number:
  US12/774,179
• Publication Number:
  US 20110001699A1
• Time in Patent Office: 2,078 Days
• Field of Search: 348/8, 345/156, 455/566
• US Class Current: 1/1
• CPC Class Codes: G06F 1/163 Wearable computers, e.g. on a belt
• CPC Class Codes: G06F 2200/1637 Sensing arrangement for detection of housing movement or orientation, e.g. for controlling scrolling or cursor movement on the display of an handheld computer
• CPC Class Codes: G06F 3/012 Head tracking input arrangements
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0304 Detection arrangements using opto-electronic means constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback

Abstract

A remote control microdisplay device that uses hand and head movement and voice commands to control the parameters of a field of view for the microdisplay within a larger virtual display area associated with a host application.

Claims

1. A headset computer comprising:

a high-resolution microdisplay, for displaying visual information received from a local processor;

a motion detector, for detecting physical movement of a body part of a user, and to provide a motion input; and

the local processor located in the headset computer, operatively coupled to the high-resolution microdisplay and the motion detector, and further comprising;

a receiver, for locally receiving the motion input;

a translator, for translating, at the local processor, the motion input to a user command, the user command being executable by a host processor to control a host display, the host processor and host display being separated from the headset computer;

wherein the user command is processed by the local processor to control an aspect of visual information displayed on the high-resolution microdisplay and controls any of;

display field of view, navigation in a display view, display zoom/pan/scale format, and selection in display view;

a communication interface, for forwarding the user command to the host processor, and for receiving information from the host processor to be displayed at the high-resolution microdisplay in response to the user command;

wherein the communication interface is a direct wireless link between the headset computer and the host processor; and

a display controller, for forwarding the information to be displayed on the high-resolution microdisplay in correspondence with display output at the host display, the information to be displayed further including at least a visual confirmation of the local processor having processed the motion input corresponding to the user command.

2. The headset computer of claim 1 further comprising:

a microphone, for receiving audio signals from the user; and

wherein the local processor further comprises;

a speech recognizer, for locally processing the audio signals to produce a voice command; and

wherein the translator further combines the voice command with the motion input to determine the user command.

3. The headset computer of claim 1 wherein the motion detector provides two or more motion inputs indicative of motion in two or more dimensions.

4. The headset computer of claim 1 wherein the motion detector is a camera for detecting hand movement of the user.

5. The headset computer of claim 1 wherein the motion detector is a head movement tracking device for detecting head movement of the user.

6. The headset computer of claim 1, wherein the user command controls a field of view.

7. The headset computer of claim 1, wherein the user command

controls a zoom, pan, or scale factor.

8. The headset computer of claim 1, wherein the user command selects an item.

9. The headset computer of claim 8 wherein the item is a hyperlink.

10. The headset computer of claim 1 wherein the reply results in a cursor movement.

17. The headset computer of claim 1 wherein the display of the information on the high resolution microdisplay is substantially simultaneous with display of said information at the host display.

11. A method for operating a headset computer having a high resolution microdisplay, a motion detector, a wireless communication interface, and a local processor, comprising:

displaying visual information received from the local processor on the high-resolution microdisplay, the local processor located in the headset computer and operatively coupled to the high-resolution microdisplay;

using the motion detector for detecting physical movement of a body part of a user as a motion input to the local processor, the local processor operatively coupled to the motion detector;

locally translating the motion input to a user command, the user command being executable by a host processor to control a host display, the user command processed by the local processor to control an aspect of visual information displayed on the high-resolution microdisplay;

forwarding the user command to the host processor directly using the wireless communications interface;

receiving visual information from the host processor to be displayed at the high-resolution microdisplay in response to the user command; and

displaying the visual information on the high-resolution microdisplay in correspondence with display output at the host display, the visual information further including at least a visual confirmation of the local processor having processed the motion input corresponding to the user command.

12. The method of claim 11 wherein one aspect of the visual information is a field of view.

13. The method of claim 11 wherein one aspect of the visual information is a zoom, pan, or scale factor.

14. The method of claim 11 wherein the motion detector provides two or more motion inputs indicative of motion of the user in two or more dimensions.

15. The method of claim 11 wherein the motion detector is a camera for detecting hand movement of the user.

16. The method of claim 11 wherein the motion detector is a head movement tracking device for detecting head movement of the user.

18. The method of claim 11 wherein the displaying of the information on the high resolution microdisplay is substantially simultaneous with display of said information at the host display.