Imbuing artificial intelligence systems with idiomatic traits

• US 9,601,104 B2
• Filed: 08/02/2016
• Issued: 03/21/2017
• Est. Priority Date: 03/27/2015
• Status: Expired due to Fees

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Cecchi, Guillermo A., Kozloski, James R., Pickover, Clifford A., Rish, Irina
• Primary Examiner(s): Pham, Thierry L
• Application Number:
  US15/226,006
• Publication Number:
  US 20160343367A1
• Time in Patent Office: 231 Days
• Field of Search: 704/220, 704/235, 704/258, 704/260, 704/275
• US Class Current: 1/1
• CPC Class Codes: G10L 13/033 Voice editing, e.g. manipulating the voice of the synthesiser
• CPC Class Codes: G10L 13/04 Details of speech synthesis systems, e.g. synthesiser structure or memory management
• CPC Class Codes: G10L 13/08 Text analysis or generation of parameters for speech synthesis out of text, e.g. grapheme to phoneme translation, prosody generation or stress or intonation determination

Abstract

Speech traits of an entity imbue an artificial intelligence system with idiomatic traits of persons from a particular category. Electronic units of speech are collected from an electronic stream of speech that is generated by a first entity. Tokens from the electronic stream of speech are identified, where each token identifies a particular electronic unit of speech from the electronic stream of speech, and where identification of the tokens is semantic-free. Nodes in a first speech graph are populated with the tokens to develop a first speech graph having a first shape. The first shape is matched to a second shape of a second speech graph from a second entity in a known category. The first entity is assigned to the known category, and synthetic speech generated by an artificial intelligence system is modified based on the first entity being assigned to the known category.

Claims

1. A method of imbuing an artificial intelligence system with idiomatic traits, the method comprising:

collecting, by one or more processors, electronic units of speech from an electronic stream of speech, wherein the electronic stream of speech is generated by a first entity;

identifying, by one or more processors, tokens from the electronic stream of speech, wherein each token identifies a particular electronic unit of speech from the electronic stream of speech, and wherein identification of the tokens is semantic-free such that the tokens are identified independently of a semantic meaning of a respective electronic unit of speech;

populating, by one or more processors, nodes in a first speech graph with the tokens;

identifying, by one or more processors, a first shape of the first speech graph;

matching, by one or more processors, the first shape to a second shape, wherein the second shape is of a second speech graph from a second entity in a known category;

assigning, by one or more processors, the first entity to the known category in response to the first shape matching the second shape;

modifying, by one or more processors, synthetic speech generated by an artificial intelligence system based on the first entity being assigned to the known category, wherein said modifying imbues the artificial intelligence system with idiomatic traits of persons in the known category; and

incorporating, by one or more processors, the artificial intelligence system with the idiomatic traits of persons in the known category into a robotic device in order to align the robotic device with cognitive traits of the persons in the known category.

2. The method of claim 1, further comprising:

defining, by one or more processors, the first shape of the first speech graph according to a size of the first speech graph, a quantity of loops in the first speech graph, sizes of the loops in the first speech graph, distances between nodes in the first speech graph, and a level of branching between the nodes in the first speech graph.

3. The method of claim 1, wherein the first entity is a person, wherein the electronic stream of speech is an electronic recording of a stream of spoken words from the person, and wherein the method further comprises:

receiving, by one or more processors, a physiological measurement of the person from a sensor, wherein the physiological measurement is taken while the person is speaking the spoken words;

analyzing, by one or more processors, the physiological measurement of the person to identify a current emotional state of the person;

modifying, by one or more processors, the first shape of the first speech graph according to the current emotional state of the person; and

further modifying, by one or more processors, the synthetic speech generated by the artificial intelligence system based on the current emotional state of the person according to the modified first shape.

4. The method of claim 1, wherein the first entity is a group of persons, wherein the electronic stream of speech is a stream of written texts from the group of persons, and wherein the method further comprises:

analyzing, by one or more processors, the written texts from the group of persons to identify an emotional state of the group of persons;

modifying, by one or more processors, the first shape of the first speech graph according to the emotional state of the group of persons; and

adjusting, by one or more processors, the synthetic speech based on a modified first shape of the first speech graph of the group of persons.

5. The method of claim 1, wherein the first entity is a person, wherein the electronic stream of speech is composed of words spoken by the person, and wherein the method further comprises:

generating, by one or more processors, a syntactic vector ({right arrow over (w)}_syn) of the words, wherein the syntactic vector describes a lexical class of each of the words;

creating, by one or processors, a hybrid graph (G) by combining the first speech graph and a semantic graph of the words spoken by the person, wherein the hybrid graph is created by;

converting, by one or more processors operating as a semantic analyzer, the words into semantic vectors, wherein a semantic similarity (sim(a,b)) between two words a and b are estimated by a scalar product (·

) of their respective semantic vectors ({right arrow over (w)}_a·

{right arrow over (w)}_b), such that;

sim(a,b)={right arrow over (w)}_a·

{right arrow over (w)}_b;

creating, by one or more processors, the hybrid graph (G) of the first speech graph and the semantic graph, where;

G={N,E,{right arrow over (W)}}

wherein N are nodes, in the hybrid graph, that represent words, E represents edges that represent temporal precedence in the electronic stream of speech, and {right arrow over (W)} is a feature vector, for each node in the hybrid graph, and wherein {right arrow over (W)} is defined as a direct sum of the syntactic vector ({right arrow over (w)}_syn) and semantic vectors ({right arrow over (w)}_sem), plus an additional direct sum of non-textual features ({right arrow over (w)}_ntxt) of the person speaking the words, such that;

{right arrow over (W)}={right arrow over (w)}_syn⊕

{right arrow over (w)}_sem⊕

{right arrow over (w)}_ntxt; and

further adjusting, by one or more processors, the synthetic speech based on a shape of the hybrid graph (G).

6. The method of claim 1, wherein the electronic stream of speech comprises spoken non-language gestures from the first entity.

7. The method of claim 1, wherein the known category is a demographic group.

8. The method of claim 1, wherein the known category is an occupational group.

9. The method of claim 1, wherein the known category is for a group having a common level of education.

10. A computer program product for imbuing an artificial intelligence system with idiomatic traits, the computer program product comprising a tangible computer readable storage medium having program code embodied therewith, wherein the program code is readable and executable by a processor to perform a method comprising:

collecting electronic units of speech from an electronic stream of speech, wherein the electronic stream of speech is generated by a first entity;

identifying tokens from the electronic stream of speech, wherein each token identifies a particular electronic unit of speech from the electronic stream of speech, and wherein identification of the tokens is semantic-free such that the tokens are identified independently of a semantic meaning of a respective electronic unit of speech;

populating nodes in a first speech graph with the tokens;

identifying a first shape of the first speech graph;

matching the first shape to a second shape, wherein the second shape is of a second speech graph from a second entity in a known category;

assigning the first entity to the known category in response to the first shape matching the second shape;

modifying synthetic speech generated by an artificial intelligence system based on the first entity being assigned to the known category, wherein said modifying imbues the artificial intelligence system with idiomatic traits of persons in the known category; and

incorporating the artificial intelligence system with the idiomatic traits of persons in the known category into a robotic device in order to align the robotic device with cognitive traits of the persons in the known category.

11. The computer program product of claim 10, wherein the method further comprises:

defining the first shape of the first speech graph according to a size of the first speech graph, a quantity of loops in the first speech graph, sizes of the loops in the first speech graph, distances between nodes in the first speech graph, and a level of branching between the nodes in the first speech graph.

12. The computer program product of claim 10, wherein the first entity is a person, wherein the electronic stream of speech is a stream of spoken words from the person, and wherein the method further comprises:

receiving a physiological measurement of the person from a sensor, wherein the physiological measurement is taken while the person is speaking the spoken words;

analyzing the physiological measurement of the person to identify a current emotional state of the person;

modifying the first shape of the first speech graph according to the current emotional state of the person; and

further modifying the synthetic speech generated by the artificial intelligence system based on the current emotional state of the person according to the modified first shape.

13. The computer program product of claim 10, wherein the first entity is a group of persons, wherein the electronic stream of speech is a stream of written texts from the group of persons, and wherein the method further comprises:

analyzing the written texts from the group of persons to identify a current emotional state of the group of persons;

modifying the first shape of the first speech graph according to the current emotional state of the group of persons; and

adjusting the synthetic speech based on a modified first shape of the first speech graph of the group of persons.

14. The computer program product of claim 10, wherein the first entity is a person, wherein the electronic stream of speech is composed of words spoken by the person, and wherein the method further comprises:

generating a syntactic vector ({right arrow over (w)}_syn) of the words, wherein the syntactic vector describes a lexical class of each of the words;

creating a hybrid graph (G) by combining the first speech graph and a semantic graph of the words spoken by the person, wherein the hybrid graph is created by;

converting the words into semantic vectors, wherein a semantic similarity (sim(a,b)) between two words a and b are estimated by a scalar product (·

) of their respective semantic vectors ({right arrow over (w)}_a·

{right arrow over (w)}_b), such that;

sim(a,b)={right arrow over (w)}_a·

{right arrow over (w)}_b; and

creating the hybrid graph (G) of the first speech graph and the semantic graph, where;

G={N,E,{right arrow over (W)}}

wherein N are nodes, in the hybrid graph, that represent words, E represents edges that represent temporal precedence in the electronic stream of speech, and {right arrow over (W)} is a feature vector, for each node in the hybrid graph, and wherein {right arrow over (W)} is defined as a direct sum of the syntactic vector ({right arrow over (w)}_syn) and semantic vectors ({right arrow over (w)}_sem), plus an additional direct sum of non-textual features ({right arrow over (w)}_ntxt) of the person speaking the words, such that;

{right arrow over (W)}={right arrow over (w)}_syn⊕

{right arrow over (w)}_sem⊕

{right arrow over (w)}_ntxt; and

further adjusting the synthetic speech based on a shape of the hybrid graph (G).

15. The computer program product of claim 10, wherein the electronic stream of speech comprises spoken non-language gestures from the first entity.

16. The computer program product of claim 10, wherein the known category is a demographic group.

17. The computer program product of claim 10, wherein the known category is an occupational group.

18. The computer program product of claim 10, wherein the known category is for a group having a common level of education.

19. A computer system comprising:

a processor, a computer readable memory, and a tangible computer readable storage medium;

first program instructions to collect electronic units of speech from an electronic stream of speech, wherein the electronic stream of speech is generated by a first entity;

second program instructions to identify tokens from the electronic stream of speech, wherein each token identifies a particular electronic unit of speech from the electronic stream of speech, and wherein identification of the tokens is semantic-free such that the tokens are identified independently of a semantic meaning of a respective electronic unit of speech;

third program instructions to populate nodes in a first speech graph with the tokens;

fourth program instructions to identify a first shape of the first speech graph;

fifth program instructions to match the first shape to a second shape, wherein the second shape is of a second speech graph from a second entity in a known category;

sixth program instructions to assign the first entity to the known category in response to the first shape matching the second shape;

seventh program instructions to modify synthetic speech generated by an artificial intelligence system based on the first entity being assigned to the known category, wherein said modifying imbues the artificial intelligence system with idiomatic traits of persons in the known category; and

eighth program instructions to incorporate the artificial intelligence system with the idiomatic traits of persons in the known category into a robotic device in order to align the robotic device with cognitive traits of the persons in the known category; and

wherein the first, second, third, fourth, fifth, sixth, seventh, and eighth program instructions are stored on the tangible computer readable storage medium and executed by the processor via the computer readable memory.

20. The computer system of claim 19, further comprising:

ninth program instructions to define the first shape of the first speech graph according to a size of the first speech graph, a quantity of loops in the first speech graph, sizes of the loops in the first speech graph, distances between nodes in the first speech graph, and a level of branching between the nodes in the first speech graph; and

wherein

the ninth program instructions are stored on the tangible computer readable storage medium and executed by the processor via the computer readable memory.