System and method for hazard mitigation in voice-driven control applications

• US 8,224,651 B2
• Filed: 04/18/2008
• Issued: 07/17/2012
• Est. Priority Date: 09/26/2006
• Status: Active Grant

• Current Assignee: Storz Endoskop Produktions GmbH (Karl Storz Se & Co. Kg)
• Original Assignee: Storz Endoskop Produktions GmbH (Karl Storz Se & Co. Kg)
• Inventors: Wang, Gang, Zheng, Chengyi, Stiller, Heinz-Werner, Contolini, Matteo
• Primary Examiner(s): Smits, Talivaldis Ivars
• Assistant Examiner(s): ROBERTS, SHAUN A
• Application Number:
  US12/105,827
• Publication Number:
  US 20090030695A1
• Time in Patent Office: 1,551 Days
• Field of Search: 704/275, 704/270
• US Class Current: 704/275
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 15/26 Speech to text systems G10L15/08 takes precedence
• CPC Class Codes: G10L 2015/223 Execution procedure of a spoken command
• CPC Class Codes: G10L 2015/228 of application context
• CPC Class Codes: G10L 25/51 for comparison or discrimination

Abstract

A speech recognition and control system including a sound card for receiving speech and converting the speech into digital data, the sound card removably connected to an input of a computer, recognizer software executing on the computer for interpreting at least a portion of the digital data, event detection software executing on the computer for detecting connectivity of the sound card, and command control software executing on the computer for generating a command based on at least one of the digital data and the connectivity of the sound card.

Claims

1. A speech recognition and control system, comprising:

a sound card for receiving speech and converting the speech into digital data, said sound card removably connected to an external input of a computer;

a microphone removably connected to said sound card;

recognizer software executing on the computer for interpreting at least a portion of the digital data;

event detection software executing on the computer for detecting connectivity of said sound card and said microphone and for detecting an event, the event being an utterance start, an utterance end, or an audio input duration exceeding a predetermined duration;

a database including a plurality of rules, the plurality of rules being predefined rules that are dependent on a current system status, and the plurality of rules comprising one or more rules to implement an action or to stop an action upon detecting the disconnection of said sound card, and the plurality of rules having one rule that provides for any system or device activity to be stopped if the utterance is begun within a predetermined period of time after the activity was commenced;

command control software executing on the computer for generating at least one command based on at least one of the digital data, the connectivity of at least one of said sound card and said microphone and at least one rule;

a plurality of devices controllable by the system; and

wherein, if a disconnection of said sound card or said microphone is detected, the at least one command generated by said command control software directs at least one of said devices controllable by the system to stop an action in progress or commence an action to mitigate a hazard without user intervention.

2. A speech recognition and control system, comprising:

a sound card for receiving speech and converting the speech into digital data, said sound card removably connected to an external input of a computer;

recognizer software executing on the computer for interpreting at least a portion of the digital data;

event detection software executing on the computer for detecting connectivity of said sound card;

a plurality of devices controllable by the system;

a database including a plurality of rules, the plurality of rules comprising rules to implement an action or stop an action of at least one of said devices upon detecting a disconnection of said sound card;

command control software executing on the computer for generating device commands based on at least one of the digital data, the plurality of rules, and the connectivity of said sound card;

wherein, if said event detection software detects the disconnection of said sound card, said command control software generates at least one device command directing at least one of said devices controllable by the system to stop an action in progress or commence an action to mitigate a hazard without user intervention.

3. The system according to claim 2, wherein the input is a universal serial bus input.

4. The system according to claim 2, further comprising software for generating an alert if the disconnection of said sound card is detected.

5. The system according to claim 4, further comprising:

a user interface including a monitor;

wherein the alert is at least one of an audible alert and a visual alert communicated via the monitor.

6. The system according to claim 2, wherein said event detection software further detects a malfunction of said sound card.

7. The system according to claim 2, wherein the at least one command further directs a second one of said devices to commence an action if a disconnection of said sound card is detected.

8. The system according to claim 2, further comprising:

a microphone removably connected to said sound card.

9. The system according to claim 8, wherein said event detection software further detects connectivity of said microphone.

10. The system according to claim 2, wherein said event detection software further detects an event, the event being an utterance start, an utterance end, or an audio input duration exceeding a predetermined duration.

11. The system according to claim 10, wherein said database further comprises at least one rule that provides for an activity of at least one of said devices to be stopped if the utterance is begun within a predetermined period of time after the activity was commenced.

12. The system according to claim 2, wherein said at least one devices directed to stop an action in progress or commence an action to mitigate a hazard without user intervention include at least one of an endoscope and a surgical table.

13. A method of mitigating hazards in a voice-driven control system, comprising the steps of:

removably connecting a sound card external to a computer via an input;

receiving speech via the sound card;

converting the speech into digital data;

determining connectivity of the sound card;

determining a system status;

determining whether a hazard mitigation action is necessary based on at least one of the digital data, the connectivity of the sound card, and at least one rule provided by a database including a plurality of rules, the plurality of rules comprising rules to implement an action or stop an action of at least one of a plurality of devices connected to the voice-driven control system upon detecting a disconnection of said sound card;

generating at least one command based on at least one of the digital data, the connectivity of the sound card, and the at least one rule, wherein, if a disconnection of said sound card is detected, said at least one command directs at least one of the devices connected to the voice-driven control system to stop an action in progress or commence an action to mitigate a hazard without user intervention; and

sending the at least one command to the at least device connected to the voice-driven control system.

14. The method according to claim 13, wherein the input is a universal serial bus input.

15. The method according to claim 13, further comprising the step of:

generating an alert if a disconnection of the sound card is detected.

16. The method according to claim 15, wherein the alert is at least one of an audible alert and a visual alert communicated via a monitor.

17. The method according to claim 13, wherein the at least one command further directs a second device connected to the voice-driven control system to commence an action if a disconnection of the sound card is detected.

18. The method according to claim 13, further comprising the step of determining connectivity of a microphone removeably connected to the sound card.

19. The method according to claim 13, wherein said at least one devices directed to stop an action in progress or commence an action to mitigate a hazard without user intervention includes a surgical table.