Advanced remote control of host application using motion and voice commands

• US 9,122,307 B2
• Filed: 09/16/2011
• Issued: 09/01/2015
• Est. Priority Date: 09/20/2010
• Status: Active Grant

• Current Assignee: Kopin Corporation
• Original Assignee: Kopin Corporation
• Inventors: Jacobsen, Jeffrey J., Parkinson, Christopher, Pombo, Stephen A.
• Primary Examiner(s): Hoang, Phi
• Application Number:
  US13/234,916
• Publication Number:
  US 20120236025A1
• Time in Patent Office: 1,446 Days
• Field of Search: 709/217, 345/8
• US Class Current: 1/1
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/012 Head tracking input arrangements
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/04817 using icons graphical or visual programming using iconic symbols G06F8/34
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: G06T 11/60 Editing figures and text; Combining figures or text

Abstract

A head mounted microdisplay (HMD) device that uses hand, body, head movement or gesture and voice commands to control interaction with a local processor or host computer. The local processor overlays information, such as onto an operating system desktop or application program menu, indicating the motion, gesture, and/or voice input needed to activate an associated action.

Claims

1. A hand-portable device comprising:

a) a microdisplay, for displaying visual information received from a local processor;

b) a motion detector, for detecting physical movement of a body part of a user, and to provide a motion or gesture input; and

c) the local processor located in the hand-portable apparatus and further comprising;

d) a transceiver, receiving the motion or gesture input from the motion detector;

e) a translator, responsive to the transceiver and translating the received motion or gesture input to a user command;

f) an icon information overlay generator, for (a) interrogating the status of a desktop to detect a displayed icon of an application program, for (b) determining overlay text or graphics associated with the displayed icon through consultation with a table, the table associating the displayed icon with the overlay text or graphics; and

(c) generating on the desktop the determined overlay text or graphics associated with the displayed icon of the application program, the generated overlay text or graphics indicating one or more possible touch-free motion or gesture inputs for activating an action of the associated application program, the application program being exclusive from a calibration program; and

g) a display controller, for forwarding information to be displayed on the microdisplay, including forwarding text content to be displayed at the information overlay.

2. The apparatus of claim 1 wherein the hand-portable device is a head mounted display (HMD) apparatus.

3. The apparatus of claim 1 wherein the information to be displayed further comprises a visual or audio confirmation of the local processor having processed the user command.

4. The apparatus of claim 1 further comprising:

a) a wireless communication interface, for forwarding user commands to a host processor, and for receiving a reply from the host processor.

5. The apparatus of claim 2 wherein the motion detector is head-tracking circuitry or a camera for detecting the motion or gesture input, the motion or gesture input being a user'"'"'s head tilt or hand motion, respectively.

6. The apparatus of claim 1 wherein the microdisplay is further configured to display visual information of the application program received from the local processor executing the application program;

and

a) wherein the icon information overlay generator is further configured to interface with the application program running on the local processor to generate the information overlay.