Device for extracting information from a dialog

• US 10,606,942 B2
• Filed: 04/25/2019
• Issued: 03/31/2020
• Est. Priority Date: 03/08/2012
• Status: Active Grant

• Current Assignee: Meta Platforms, Inc. (f/k/a Facebook, Inc.)
• Original Assignee: Meta Platforms, Inc. (f/k/a Facebook, Inc.)
• Inventors: Waibel, Alexander
• Primary Examiner(s): Pham, Thierry L
• Application Number:
  US16/395,001
• Publication Number:
  US 20190251156A1
• Time in Patent Office: 341 Days
• Field of Search: 704 2, 704 4, 704 9, 704211, 704246, 704277
• CPC Class Codes: G06F 3/04842 Selection of displayed objects or displayed text elements G06F3/0482 takes precedence
• CPC Class Codes: G06F 3/0486 Drag-and-drop
• CPC Class Codes: G06F 40/117 Tagging; Marking up details of markup languages G06F40/143; Designating a block; Setting of attributes style sheets, e.g. eXtensible Stylesheet Language Transformation [XSLT], G06F40/154
• CPC Class Codes: G06F 40/174 Form filling; Merging
• CPC Class Codes: G06F 40/58 Use of machine translation, e.g. for multi-lingual retrieval, for server-side translation for client devices or for real-time translation
• CPC Class Codes: G10L 13/027 Concept to speech synthesisers; Generation of natural phrases from machine-based concepts generation of parameters for speech synthesis out of text G10L13/08
• CPC Class Codes: G10L 15/00 Speech recognition G10L17/00 takes precedence
• CPC Class Codes: G10L 15/1815 Semantic context, e.g. disambiguation of the recognition hypotheses based on word meaning
• CPC Class Codes: G10L 15/1822 Parsing for meaning understanding
• CPC Class Codes: G10L 15/26 Speech to text systems G10L15/08 takes precedence
• CPC Class Codes: G10L 2015/088 Word spotting
• CPC Class Codes: G16H 10/20 for electronic clinical trials or questionnaires
• CPC Class Codes: G16H 15/00 ICT specially adapted for medical reports, e.g. generation or transmission thereof
• CPC Class Codes: G16H 80/00 ICT specially adapted for facilitating communication between medical practitioners or patients, e.g. for collaborative diagnosis, therapy or health monitoring

Abstract

Computer-implemented systems and methods for extracting information during a human-to-human mono-lingual or multi-lingual dialog between two speakers are disclosed. Information from either the recognized speech (or the translation thereof) by the second speaker and/or the recognized speech by the first speaker (or the translation thereof) is extracted. The extracted information is then entered into an electronic form stored in a data store.

Claims

1. A system, comprising:

a speaker configured to receive a first speech input in a first language from a first speaker and a second speech input in a second language from a second speaker, the first speech input and the second speech input being part of a dialogue between the first speaker and the second speaker;

a display configured to display a form having a plurality of fields;

a processor; and

a memory storing instructions, when executed by the processor, cause the processor to;

determine a first recognized speech result based on the first speech input;

determine a second recognized speech result based on the second speech input;

extract information from the second recognized speech result;

identify one of the plurality of fields based on the first recognized speech result in the first language;

translate at least a portion of the information extracted from the second recognized speech result from the second language to the first language;

insert the translated information in the first language into the identified field; and

cause the display to present the form in the first language.

2. The system of claim 1, wherein the form comprises a toggle switch that is configured to cause the form to change from the first language to the second language or from the second language to the first language.

3. The system of claim 1, wherein the display is capable of receiving one or more touch inputs from a user adjusting the form.

4. The system of claim 3, wherein the instructions, when executed by the processor, further cause the processor to:

receive a correction to the form by detecting one of the touch inputs from the user dragging a part of the information from the second recognized speech result to an empty field of the form; and

enter the dragged part into the empty field.

5. The system of claim 1, wherein the instructions, when executed by the processor, further cause the processor to:

receive a selection of an empty field of the form;

receive a correction speech input;

determine a recognized correction speech result based on the correction speech input; and

enter the recognized correction speech result into the empty field.

6. The system of claim 1, wherein the instructions, when executed by the processor, further cause the processor to:

receive selection of a filled field of the form;

receive selection of a portion of the second recognized speech result; and

replace contents of the filled field with the portion of the recognized speech result.

7. The system of claim 1, wherein the instructions, when executed by the processor, further cause the processor to:

receiving a selection of a filled field of the form;

receiving a correction speech input;

determining a recognized correction speech result based on the correction speech input; and

replacing contents of the filled field with the recognized correction speech result.

8. The system of claim 7, wherein the correction speech input is a dictated spelling.

9. The system of claim 7, wherein the correction speech input is a restatement of at least a portion of the second speech input.

10. The system of claim 1, wherein the information is extracted based on one or more prompts associated with the plurality of fields of the form.

11. The system of claim 1, wherein the information is extracted using a semantic parser.

12. The system of claim 1, wherein the information is extracted using keyword spotting.

13. The system of claim 1, wherein the information is extracted via spoken term detection.

14. The system of claim 1, wherein the information is extracted using an entity tagger.

15. A portable electronic device, comprising:

a processor; and

a memory storing instructions, when executed by the processor, cause the processor to;

receive a first speech input in a first language from a first speaker and a second speech input in a second language from a second speaker, the first speech input and the second speech input being part of a dialogue between the first speaker and the second speaker;

determine a first recognized speech result based on the first speech input;

determine a second recognized speech result based on the second speech input;

extract information from the second recognized speech result;

identify one of a plurality of fields of a form based on the first recognized speech result in the first language;

translate at least a portion of the information extracted from the second recognized speech result from the second language to the first language;

insert the translated information in the first language into the identified field; and

cause a display of the form in the first language.

16. The portable electronic device of claim 15, wherein the form comprises a toggle switch that is configured to cause the form to change from the first language to the second language or from the second language to the first language.

17. The portable electronic device of claim 15, wherein the instructions, when executed by the processor, further cause the processor to:

receive a correction to the form by detecting a touch input from a user dragging a part of the information from the second recognized speech result to an empty field of the form; and

enter the dragged part into the empty field.

18. The portable electronic device of claim 15, wherein the instructions, when executed by the processor, further cause the processor to:

receive a selection of an empty field of the form;

receive a correction speech input;

determine a recognized correction speech result based on the correction speech input; and

enter the recognized correction speech result into the empty field.

19. The portable electronic device of claim 15, wherein the instructions, when executed by the processor, further cause the processor to:

receive selection of a filled field of the form;

receive selection of a portion of the second recognized speech result; and

replace contents of the filled field with the portion of the recognized speech result.

20. The portable electronic device of claim 15, wherein the instructions, when executed by the processor, further cause the processor to:

receiving a selection of a filled field of the form;

receiving a correction speech input;

determining a recognized correction speech result based on the correction speech input; and

replacing contents of the filled field with the recognized correction speech result.