Data processing method, presentation method, and corresponding apparatuses

• US 9,158,753 B2
• Filed: 07/16/2013
• Issued: 10/13/2015
• Est. Priority Date: 07/12/2012
• Status: Expired due to Fees

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Bao, Sheng Hua, Chen, Jian, Liu, Wen, Qin, Yong, Shi, Qin, Su, Zhong, Zhang, Shi Lei
• Primary Examiner(s): Chawan, Vijay B
• Application Number:
  US13/943,308
• Publication Number:
  US 20140019121A1
• Time in Patent Office: 819 Days
• Field of Search: 704 2- 9, 704/270, 704/270.1, 704/257, 704/158, 704/235, 704/240, 704/251, 704/253, 704/255, 704/239, 704/238, 704/237, 704/243, 715/201, 715/245, 715/203, 715/256, 715/234, 381 8801- 8804, 709/219, 709/206
• US Class Current: 1/1
• CPC Class Codes: G06F 40/20 Natural language analysis semantic analysis of natural language G06F40/30
• CPC Class Codes: G06F 40/279 Recognition of textual entities
• CPC Class Codes: G10L 15/18 using natural language modelling
• CPC Class Codes: G10L 15/187 Phonemic context, e.g. pronunciation rules, phonotactical constraints or phoneme n-grams
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue

Abstract

A data processing method includes obtaining text information corresponding to a presented content, the presented content comprising a plurality of areas; performing text analysis on the text information to obtain a first keyword sequence, the first keyword sequence including area keywords associated with at least one area of the plurality of areas; obtaining speech information related to the presented content, the speech information at least comprising a current speech segment; and using a first model network to perform analysis on the current speech segment to determine the area corresponding to the current speech segment, wherein the first model network comprises the first keyword sequence.

Claims

1. An apparatus for data processing to perform text and speech analysis of a presentation, the apparatus comprising:

a processor coupled to a system bus and operable to access one or more of a serial interface controller and a parallel interface controller, the processor operable to execute instructions to perform a method comprising;

obtaining, by the processor, text information corresponding to a presented content, the presented content comprising a plurality of areas, each of the areas comprising a segment of the presented content partitioned according to one or more criteria;

performing text analysis, by the processor, on the text information to obtain a first keyword sequence, the first keyword sequence including area keywords associated with at least one area of the plurality of areas;

obtaining speech information related to the presented content, the speech information at least comprising a current speech segment of a presenter during a presentation and the speech information is obtained by the processor over the system bus from one or more of the serial interface controller or the parallel interface controller as audio or video of the presenter during the presentation;

using a first model network to perform analysis on the current speech segment by the processor to determine the area corresponding to the current speech segment, wherein the first model network comprises the first keyword sequence;

obtaining a plurality of second keyword sequences by the processor, at least one of the second keyword sequences corresponding to at least one area of the plurality of areas and at least one of the second keyword sequences comprising at least one keyword;

using a second model network to perform analysis on the current speech segment by the processor to determine a keyword corresponding to the current speech segment, the second model network comprising the second keyword sequences; and

marking the presented content, by the processor, to attract audience attention based on determining the keyword corresponding to the current speech segment.

2. The apparatus according to claim 1, wherein the using a first model network to perform analysis on the current speech segment to determine the area corresponding to the current speech segment comprises:

obtaining a confidence degree of at least one area keyword in the first keyword sequence, wherein the higher similarity an area keyword has with respect to the current speech segment, the higher confidence degree is obtained for the area keyword;

determining, if a first condition is met, that the area corresponding to the current speech segment is an area associated with an area keyword involved in the first condition;

wherein the first condition comprises at least one of the following;

that a confidence degree of an area keyword reaches a first threshold;

that all confidence degrees of multiple area keywords associated with a same area reach a second threshold;

that a sum of confidence degrees of multiple area keywords associated with a same area reaches a third threshold.

3. The apparatus according to claim 2,

wherein the first model network further comprises a first phoneme sequence;

wherein the using a first model network to perform analysis on the current speech segment to determine the area corresponding to the current speech segment further comprises;

obtaining at least one phoneme adjacent to the current speech segment based on the first phoneme sequence;

determining a pronunciation similarity between the at least one phoneme and corresponding text information of the at least one area keyword, the corresponding text information of the at least one area keyword comprising a context of the at least one area keyword in the text information;

if the pronunciation similarity between the at least one phoneme and the corresponding text information of the at least one area keyword reaches a fourth threshold, raising the confidence degree of the area keyword with the pronunciation similarity reaching the fourth threshold.

4. The apparatus according to claim 2, wherein the processor is operable to execute instructions to perform the method further comprising:

obtaining a plurality of second keyword sequences, at least one of the second keyword sequences corresponding to at least one area of the plurality of areas and at least one of the second keyword sequence comprising at least one keyword;

obtaining a confidence degree of at least one keyword in at least one of the second keyword sequences, wherein a keyword having a higher similarity with the current speech segment has a higher confidence degree;

the determining, if a first condition is met, that the area corresponding to the current speech segment is an area associated with an area keyword involved in the first condition comprises;

if the first condition is met and a second condition also is met, determining that the area corresponding to the current speech segment is the area associated with the area keyword involved in the first condition, and taking the area corresponding to the current speech segment as the current area;

wherein the second condition comprises that a confidence degree of a second keyword sequence corresponding to the current area is less than a fifth threshold, the confidence degree of the second keyword sequence corresponding to the current area being obtained based on a confidence degree of a keyword included in the second keyword sequence corresponding to the current area.

5. The apparatus according to claim 2, the processor is operable to execute instructions to perform the method further comprising:

obtaining a plurality of second keyword sequences, at least one of the second keyword sequences corresponding to at least one area of the plurality of areas and at least one of the second keyword sequences comprising at least one keyword;

using a second model network to perform analysis on the speech segment, the second model network comprising the second keyword sequences and a second phoneme sequence;

the determining, if a first condition is met, that the area corresponding to the current speech segment is an area associated with an area keyword involved in the first condition comprises;

if the first condition is met and a third condition also is met, determining that the area corresponding to the current speech segment is the area associated with the area keyword involved in the first condition;

wherein the third condition comprises that when using the second model network to perform the analysis on the current speech segment, the times of obtaining outputs based on the second phoneme sequence reaches a sixth threshold.

6. The apparatus according to claim 1, wherein using the second model network to perform analysis on the current speech segment to determine a keyword corresponding to the current speech segment is performed simultaneous with the analysis of the first model network.

7. The apparatus according to claim 1, wherein the second model network further comprises a second phoneme sequence;

wherein the using a second model network to perform analysis on the current speech segment to determine a keyword corresponding to the current speech segment comprises;

obtaining at least one phoneme adjacent to the current speech segment based on the second phoneme sequence;

determining a pronunciation similarity between the at least one phoneme and corresponding text information of the at least one keyword, the corresponding text information of the at least one keyword comprising a context of the at least one keyword in the text information; and

if the pronunciation similarity between the at least one phoneme and the corresponding text information of the at least one keyword reaches a threshold, increasing the confidence degree of the keyword with the pronunciation similarity reaching the threshold.

8. The apparatus according to claim 1, wherein the second model network further comprises a second phoneme sequence, and the processor is operable to execute instructions to perform the method further comprising:

obtaining at least one phoneme adjacent to the current speech segment based on the second phoneme sequence; and

determining a pronunciation similarity between the at least one phoneme and corresponding text information of the candidate keyword, the corresponding text information of the candidate keyword comprising a context of the candidate keyword in the text information.

9. An apparatus to automatically jump to a different area of a presentation based on text and speech analysis, the apparatus comprising:

a processor coupled to a system bus and operable to access one or more of a serial interface controller and a parallel interface controller, the processor operable to execute instructions to perform a method comprising;

obtaining, by the processor, text information corresponding to a presented content, the presented content comprising a plurality of areas, each of the areas comprising a segment of the presented content partitioned according to one or more criteria;

performing text analysis, by the processor, on the text information to obtain a first keyword sequence, the first keyword sequence comprising an area keyword associated with at least one area of the plurality of areas and obtain a plurality of second keyword sequences, at least one of the second keyword sequences corresponding to at least one area of the plurality of areas and at least one of the second keyword sequences comprising at least one keyword;

obtaining speech information related to the presented content, the speech information at least comprising a current speech segment of a presenter during a presentation and the speech information is by the processor over the system bus from one or more of the serial interface controller or the parallel interface controller as audio or video of the presenter during the presentation;

obtaining a confidence degree of at least one keyword in at least one of the keyword sequences, wherein the higher similarity a keyword has with the current speech segment, the higher confidence degree is obtained for the keyword;

obtaining the confidence degree of the second keyword sequence corresponding to a current area of the plurality of areas based on the confidence degree of the keyword; and

jumping away from the current area of the presented content by the processor in response to the confidence degree of the second keyword sequence corresponding to the current area being less than a threshold.

10. The apparatus according to claim 9, the processor is operable to execute instructions to perform the method further comprising:

obtaining a confidence degree of at least one area keyword in the first keyword sequence based on the first keyword sequence, wherein the higher similarity an area keyword has with respect to the current speech segment, the higher confidence degree is obtained for the area keyword.