Audio, Visual and device data capturing system with real-time speech recognition command and control system

• US 20080062280A1
• Filed: 09/12/2006
• Published: 03/13/2008
• Est. Priority Date: 09/12/2006
• Status: Active Grant

• Current Assignee: Storz Endoskop Produktions GmbH (Karl Storz Se & Co. Kg)
• Original Assignee: Storz Endoskop Produktions GmbH (Karl Storz Se & Co. Kg)
• Inventors: Wang, Gang, Contolini, Matteo, Zheng, Chengyi, Stiller, Heinz-Werner
• Granted Patent: US 8,502,876 B2
• Time in Patent Office: Days
• US Class Current: 348/231.99
• CPC Class Codes: G10L 15/26 Speech to text systems G10L15/08 takes precedence
• CPC Class Codes: G16H 10/60 for patient-specific data, e.g. for electronic patient records
• CPC Class Codes: G16H 30/20 for handling medical images, e.g. DICOM, HL7 or PACS
• CPC Class Codes: G16H 30/40 for processing medical images, e.g. editing

Abstract

An audio, visual and device data capturing system including an audio recorder for recording audio data, at least one visual recorder for recording visual data, at least one device data recorder for receiving device data from at least one device in communication with the system, a speech recognition module for interpreting the audio data, a transcript module for generating transcript data from the interpreted audio data, a data capturing module for generating a data record including at least a portion of each of the audio data, the transcript data, the visual data and the device data, and at least one storage device for storing the data record.

Claims

1. An audio and visual data capturing system, comprising:

an audio recorder for recording audio data;

at least one visual recorder for recording visual data;

a speech recognition module for interpreting the audio data;

a transcript module for generating transcript data from the interpreted audio data;

a data capturing module for generating a data record including at least a portion of each of the audio data, the transcript data, and the visual data; and

at least one storage device for storing the data record.

2. The system according to claim 1, further comprising:

at least one device data recorder for receiving device data from at least one device in communication with the system,

wherein the data record further includes at least a portion of the device data.

3. The system according to claim 2, wherein said data capturing module synchronizes the audio data, the transcript data, the visual data and the device data in the data record.

4. The system according to claim 1, wherein said data capturing module synchronizes the audio data, the transcript data and the visual data in the data record.

5. The system according to claim 1, wherein the audio data includes one or more voice annotations corresponding to one or more portions of the visual data.

6. The system according to claim 1, wherein said at least one storage device includes a plurality of data records, wherein the system comprises software for querying the plurality of data records based on the transcript data.

7. The system according to claim 1, wherein said transcript module generates the transcript data continuously while said audio recorder records the audio data.

8. The system according to claim 1, wherein said at least one storage device includes a database, wherein the data record is stored in a location of the database based on at least a portion of the transcript data.

9. The system according to claim 1, wherein said at least one storage device includes a medical records database.

10. The system according to claim 1, wherein said at least one storage device includes a portable media drive.

11. The system according to claim 1, wherein said at least one visual recorder includes a video image recorder and a still image recorder, wherein the visual data includes both still image data and video image data.

12. An audio and visual data capturing system with speech recognition and control, comprising:

an audio recorder for recording audio data;

at least one visual recorder for recording visual data;

a speech recognition module for interpreting the audio data and generating transcript data;

a controller for generating a control command based on at least a portion of the interpreted audio data;

a data capturing module for generating a data record including at least a portion of each of the audio data, the visual data and the transcript data; and

at least one storage device for storing the data record.

13. The system according to claim 12, wherein the audio data includes a plurality of user utterances, wherein the system further comprises a discriminator for distinguishing between user utterances intended as control commands and user utterances intended as transcript data.

14. The system according to claim 13, wherein said discriminator distinguishes based on recognition grammar.

15. The system according to claim 14, wherein the recognition grammar is updated upon at least one of a device status change and a command menu change.

16. The system according to claim 12, further comprising:

at least one device data recorder for receiving device data from at least one device in communication with the system,

wherein the data record further includes at least a portion of the device data.

17. The system according to claim 12, wherein the speech recognition module includes a transcript module for generating the transcript data.

18. The system according to claim 12, wherein said data capturing module synchronizes the audio data, the visual data and the transcript data.

19. The system according to claim 12, wherein said controller further sends the control command to a device operable by the system.

20. The system according to claim 19, wherein the device operable by the system is a medical device.

21. The system according to claim 19, wherein the device operable by the system is one of said audio recorder and said at least one visual recorder.

22. A method of processing audio, visual and device data, comprising the steps of:

recording audio data, the audio data including one or more user utterances;

recording at least one of visual data and device data;

interpreting the one or more user utterances;

generating transcript data;

generating a data record including at least a portion of each of the audio data and the transcript data; and

storing the data record.

23. The method according to claim 22, further comprising the step of:

synchronizing the audio data and the transcript data in the data record.

24. The method according to claim 22, wherein the data record further includes the visual data, wherein the method further includes the step of synchronizing the audio data, the transcript data, and the visual data in the data record.

25. The method according to claim 22, wherein the data record further includes the device data, wherein the method further includes the step of synchronizing the audio data, the transcript data, and the device data in the data record.

26. The method according to claim 22, wherein said step of interpreting the one or more user utterances includes discriminating between user utterances intended as control commands and user utterances intended for transcription.

27. The method according to claim 26, wherein said step of discriminating including querying recognition grammar, wherein the method further comprises a step of updating the recognition grammar upon one of a device status change and a command menu change.

28. The method according to claim 26, wherein said step of generating transcript data consists of transcribing the user utterances intended for transcription.

29. The method according to claim 26, further comprising the step of:

sending the user utterances intended as control commands to a command controller for execution.

30. The method according to claim 29, further comprising the step of:

sending a control command to a device operable by the control command.

31. The method according to claim 22, further comprising the steps of:

receiving a search request including at least one content term;

querying the at least a portion of the transcript data of the data record for the at least one content term; and

providing a portion of the data record corresponding to the at least one content term.

32. The method according to claim 22, wherein at least one of the user utterances corresponds to a portion of the visual data.

33. The method according to claim 22, wherein the data record is stored in a database location based on the transcript data.

34. The method according to claim 22, further comprising the step of:

generating a control command based on the interpreted audio data and sending the control command to a device operable by the control command.