Topic shift detector

• US 10,108,702 B2
• Filed: 08/24/2015
• Issued: 10/23/2018
• Est. Priority Date: 08/24/2015
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Byron, Donna K., Emanuel, Barton W., Mahdad, Samir, Rakshit, Sarbajit K., Trim, Craig M.
• Primary Examiner(s): Park, Grace
• Application Number:
  US14/833,577
• Publication Number:
  US 20170060994A1
• Time in Patent Office: 1,156 Days
• CPC Class Codes: G06F 16/3344 using natural language analysis
• CPC Class Codes: G06F 16/355 Class or cluster creation or modification
• CPC Class Codes: G10L 15/1822 Parsing for meaning understanding
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 2015/088 Word spotting
• CPC Class Codes: G10L 2015/227 of the speaker; Human-factor methodology

Abstract

Aspects detect or recognize shifts in topics in computer implemented speech recognition processes as a function of mapping keywords to non-verbal cues. An initial topic is mapped to one or more keywords extracted from a first spoken query within a user keyword ontology mapping. A query spoken subsequent in time to the first query is identified and distinguished by recognizing one or more non-verbal cues associated with the audio data input that include a time elapsed between the queries, and in some aspects a user'"'"'s facial expression or motion activity. Aspects determine whether the second spoken query is directed to the initial topic or to a new topic that is different from the initial topic, as a function of mappings of the keyword(s) extracted from the first query to one or more keywords extracted from the second query and to the non-verbal cue(s) within the user ontology mapping.

Claims

1. A computer-implemented method for detecting shifts in topics in computer implemented speech recognition processes as a function of mapping keywords to non-verbal cues, the method comprising executing on a computer processor:

identifying an initial topic of a first spoken query within an audio data input from a user that is mapped to at least one keyword extracted from the first spoken query within a user keyword ontology mapping;

identifying a second spoken query within the audio data input that is subsequent in time to the first spoken query and is distinguished from the first query by recognizing at least one non-verbal cue associated with the audio data input, wherein the at least one non-verbal cue comprises a time elapsed between the first spoken query and the second spoken query, and a user'"'"'s motion activity relative to a programmable device comprising the processor during a time between the first spoken query and the second spoken query;

determining whether the second spoken query is directed to the initial topic or to a new topic that is different from the initial topic, as a function of mappings of the at least one keyword extracted from the first query to at least one keyword extracted from the second query and to the at least one non-verbal cue within the user ontology mapping; and

determining that the second spoken query is directed to the new topic in response to determining that the user'"'"'s motion activity indicates motion away from progressing toward a geographic reference point of a result that satisfies the first spoken query.

2. The method of claim 1, further comprising:

integrating computer-readable program code into a computer system comprising a processor, a computer readable memory in circuit communication with the processor, and a computer readable storage medium in circuit communication with the processor; and

wherein the processor executes program code instructions stored on the computer-readable storage medium via the computer readable memory and thereby performs the identifying the initial topic, identifying the second spoken query within the audio data input that is subsequent in time to the first spoken query and is distinguished from the first query by recognizing the at least one non-verbal cue associated with the audio data input, determining whether the second spoken query is directed to the initial topic of the first spoken query or to a new topic that is different from the initial topic, and determining that the second spoken query is directed to the new topic in response to determining that the user'"'"'s motion activity indicates the motion away from progressing toward the geographic reference point of the result that satisfies the first spoken query.

3. The method of claim 1, further comprising executing on the processor the steps of:

generating a response to the second spoken query that satisfies;

the initial topic, in response to determining that the second spoken query is directed to the initial topic;

the new topic, in response to determining that the second spoken query is directed to the new topic and is independent of the initial topic; and

the initial topic and the new topic, in response to determining that the second spoken query is directed to the new topic within a context of the initial topic.

4. The method of claim 3, further comprising:

determining that the second spoken query is directed to the new topic in response to determining that the time elapsed between the first spoken query and the second spoken query is in excess of a threshold time period for topic continuity.

5. The method of claim 4, further comprising:

in response to determining that the time elapsed between the first and second spoken queries is shorter than a threshold contextual continuity time period, generating the response to the second spoken query to satisfy the new topic and the initial topic of the first query.

6. The method of claim 1,

further comprising:

determining that the second spoken query is directed to the new topic in response to determining that a gaze orientation of the user during the second spoken query is different from an initial gaze orientation during the first spoken query.

7. The method of claim 1, wherein the at least one non-verbal cue comprises a user'"'"'s facial expression detected during a time between the first spoken query and the second spoken query;

and

the method further comprising;

determining that the second spoken query is directed to the new topic in response to determining that the user'"'"'s facial expression during the time between the first spoken query and the second spoken query is selected from the group consisting of a frown, a scowl and a grimace.

8. A system, comprising:

a processor;

a computer readable memory in circuit communication with the processor; and

a computer readable storage medium in circuit communication with the processor;

wherein the processor executes program instructions stored on the computer-readable storage medium via the computer readable memory and thereby;

identifies an initial topic of a first spoken query within an audio data input from a user that is mapped to at least one keyword extracted from the first spoken query within a user keyword ontology mapping;

identifies a second spoken query within the audio data input that is subsequent in time to the first spoken query and is distinguished from the first query by recognizing at least one non-verbal cue associated with the audio data input, wherein the at least one non-verbal cue comprises a time elapsed between the first spoken query and the second spoken query, and a user'"'"'s motion activity relative to a programmable device comprising the processor during a time between the first spoken query and the second spoken query;

determines whether the second spoken query is directed to the initial topic or to a new topic that is different from the initial topic, as a function of mappings of the at least one keyword extracted from the first query to at least one keyword extracted from the second query and to the at least one non-verbal cue within the user ontology mapping; and

determines that the second spoken query is directed to the new topic in response to determining that the user'"'"'s motion activity indicates motion away from progressing toward a geographic reference point of a result that satisfies the first spoken query.

9. The system of claim 8, wherein the processor executes the program instructions stored on the computer-readable storage medium via the computer readable memory and thereby further:

generates a response to the second spoken query that satisfies;

the initial topic, in response to determining that the second spoken query is directed to the initial topic;

the new topic, in response to determining that the second spoken query is directed to the new topic and is independent of the initial topic; and

the initial topic and the new topic, in response to determining that the second spoken query is directed to the new topic within a context of the initial topic.

10. The system of claim 9, wherein the processor executes the program instructions stored on the computer-readable storage medium via the computer readable memory and thereby further determines that the second spoken query is directed to the new topic in response to determining that the time elapsed between the first spoken query and the second spoken query is in excess of a threshold time period for topic continuity.

11. The system of claim 10, wherein the processor executes the program instructions stored on the computer-readable storage medium via the computer readable memory and thereby further:

in response to determining that the time elapsed between the first and second spoken queries is shorter than a threshold contextual continuity time period, generates the response to the second query to satisfy the new topic and the topic of the first query.

12. The system of claim 9,

wherein the processor executes the program instructions stored on the computer-readable storage medium via the computer readable memory and thereby determines that the second spoken query is directed to the new topic in response to determining that a gaze orientation of the user during the second spoken query is different from an initial gaze orientation during the first spoken query.

13. The system of claim 9, wherein the at least one non-verbal cue comprises a user'"'"'s facial expression detected during a time between the first spoken query and the second spoken query;

and

wherein the processor executes the program instructions stored on the computer-readable storage medium via the computer readable memory and thereby determines that the second spoken query is directed to the new topic in response to determining that the user'"'"'s facial expression during the time between the first spoken query and the second spoken query is selected from the group consisting of a frown, a scowl and a grimace.

14. A computer program product for detecting shifts in topics in computer implemented speech recognition processes as a function of mapping keywords to non-verbal cues, the computer program product comprising:

a computer readable storage medium having computer readable program code embodied therewith, wherein the computer readable storage medium is not a transitory signal per se, the computer readable program code comprising instructions for execution by a processor that cause the processor to;

identify an initial topic of a first spoken query within an audio data input from a user that is mapped to at least one keyword extracted from the first spoken query within a user keyword ontology mapping;

identify a second spoken query within the audio data input that is subsequent in time to the first spoken query and is distinguished from the first query by recognizing at least one non-verbal cue associated with the audio data input, wherein the at least one non-verbal cue comprises a time elapsed between the first spoken query and the second spoken query, and a user'"'"'s motion activity relative to a programmable device comprising the processor during a time between the first spoken query and the second spoken query;

determine whether the second spoken query is directed to the initial topic or to a new topic that is different from the initial topic, as a function of mappings of the at least one keyword extracted from the first query to at least one keyword extracted from the second query and to the at least one non-verbal cue within the user ontology mapping; and

determine that the second spoken query is directed to the new topic in response to determining that the user'"'"'s motion activity indicates motion away from progressing toward a geographic reference point of a result that satisfies the first spoken query.

15. The computer program product of claim 14, wherein the computer readable program code instructions for execution by the processor further cause the processor to:

generate a response to the second spoken query that satisfies;

the initial topic, in response to determining that the second spoken query is directed to the initial topic;

the new topic, in response to determining that the second spoken query is directed to the new topic and is independent of the initial topic; and

the initial topic and the new topic, in response to determining that the second spoken query is directed to the new topic within a context of the initial topic.

16. The computer program product of claim 15, wherein the computer readable program code instructions for execution by the processor further cause the processor to determine that the second spoken query is directed to the new topic in response to determining that the time elapsed between the first spoken query and the second spoken query is in excess of a threshold time period for topic continuity.

17. The computer program product of claim 15, wherein the computer readable program code instructions for execution by the processor further cause the processor to, in response to determining that the time elapsed between the first and second queries is shorter than a threshold contextual continuity time period, generate the response to the second query to satisfy the new topic and the topic of the first query.

18. The computer program product of claim 15,

wherein the computer readable program code instructions for execution by the processor further cause the processor to determine that the second spoken query is directed to the new topic in response to determining that a gaze orientation of the user during the second spoken query is different from an initial gaze orientation during the first spoken query.