Automatic speech recognition (ASR) feedback for head mounted displays (HMD)

• US 10,209,955 B2
• Filed: 11/13/2014
• Issued: 02/19/2019
• Est. Priority Date: 11/15/2013
• Status: Active Grant

• Current Assignee: Kopin Corporation
• Original Assignee: Kopin Corporation
• Inventors: Parkinson, Christopher, Woodall, James
• Primary Examiner(s): Pham, Thierry L
• Application Number:
  US14/540,943
• Publication Number:
  US 20150142440A1
• Time in Patent Office: 1,559 Days
• Field of Search: 704249
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 17/22 Interactive procedures; Man-machine interfaces
• CPC Class Codes: G10L 2015/225 Feedback of the input speech

Abstract

Feedback mechanisms to the user of a Head Mounted Display (HMD) are provided. It is important to provide feedback to the user when speech is recognized as soon as possible after the user utters a voice command. The HMD displays and/or audibly renders an ASR acknowledgment in a manner that ensures the user that the HMD has received/understood his voiced command.

Claims

1. A method of utilizing a user input from a user, comprising:

receiving, by a headset computer, the user input comprising a combination of one or more of (i) a voice command and a head movement and (ii) a voice command and a hand gesture;

interpreting the head movement to generate an interpreted head movement command;

interpreting the hand gesture to generate an interpreted hand gesture;

interpreting the voice command using an automatic speech recognition system to generate an interpreted voice command, and providing visual feedback to the user on a display such that the visual feedback is presented to the user one of (a) within 500mS of when the voice command is interpreted and (b) within two cycles of a frame rate of the display of when the voice command is interpreted; and

combining the interpreted voice command and at least one of the interpreted head movement command and the interpreted hand gesture command to generate a host command.

2. The method of claim 1, further including converting, using a microphone, an utterance corresponding to the voice command into an electrical signal.

3. The method of claim 1, wherein interpreting the voice command includes determining if the voice command is a member of a predetermined suite of voice commands.

4. The method of claim 1, wherein acknowledging the received voice command includes presenting an automatic speech recognition (ASR) notification clue.

5. The method of claim 4, wherein the ASR notification clue includes a visual clue.

6. The method of claim 5, wherein the visual clue remains on the display for a predetermined time after the visual clue is instantiated, and removed from the display after the predetermined time has elapsed.

7. The method of claim 4, wherein the ASR notification clue includes an audio clue.

8. The method of claim 7, wherein the audio clue is a chirp.

9. An apparatus for utilizing a user input from a user, comprising:

a headset computer, including a processor, configured to;

receive the user input comprising a combination of one or more of (i) a voice command and a head movement and (ii) a voice command and a hand gesture;

interpret the head movement to generate an interpreted head movement command;

interpret the hand gesture to generate an interpreted hand gesture;

interpreting the voice command using an automatic speech recognition system to generate an interpreted voice command, and providing visual feedback to the user on a display such that the visual feedback is presented to the user one of (a) within 500mS of when the voice command is interpreted and (b) within two cycles of a frame rate of the display of when the voice command is interpreted; and

combine the interpreted voice command and at least one of the interpreted head movement command and the interpreted hand gesture command to generate a host command.

10. The apparatus of claim 9, wherein the headset computer is further configured to convert, using a microphone, an utterance corresponding to the voice command into an electrical signal.

11. The apparatus of claim 9, wherein the headset computer is further configured to determine if the voice command is a member of a predetermined suite of voice commands.

12. The apparatus of claim 9, the headset computer is further configured to present an automatic speech recognition (ASR) notification clue.

13. The apparatus of claim 12, wherein the ASR notification clue includes a visual clue.

14. The apparatus of claim 13, wherein the visual clue remains on the display for a predetermined time after the visual clue is instantiated, and removed from the display after the predetermined time has elapsed.

15. The apparatus of claim 12, wherein the ASR notification clue includes an audio clue.

16. The apparatus of claim 15, wherein the audio clue is a chirp.