Information processing apparatus and information processing method

• US 9,965,463 B2
• Filed: 10/27/2014
• Issued: 05/08/2018
• Est. Priority Date: 01/15/2014
• Status: Active Grant

• Current Assignee: Sony Corporation (Sony Group Corp.)
• Original Assignee: Sony Corporation (Sony Group Corp.)
• Inventors: Tsunokawa, Motoki
• Primary Examiner(s): SHIN, SEONG-AH A
• Application Number:
  US15/109,841
• Publication Number:
  US 20160328379A1
• Time in Patent Office: 1,289 Days
• Field of Search: 704 9, 704224, 704235, 704257
• CPC Class Codes: G06F 16/433 using audio data
• CPC Class Codes: G06F 16/634 Query by example, e.g. query by humming
• CPC Class Codes: G06F 40/205 Parsing
• CPC Class Codes: G06F 40/268 Morphological analysis
• CPC Class Codes: G06F 40/35 Discourse or dialogue representation
• CPC Class Codes: G06N 5/022 Knowledge engineering; Knowledge acquisition

Abstract

An information processing apparatus includes: an intention understanding unit configured to understand intention of a question inputted from a user; and an information extracting unit configured to extract response basic information which is a response to the question understood by the intention understanding unit and response addition information having relationship with the response basic information from a plurality of pieces of knowledge information including knowledge information relating to the user. Each piece of knowledge information includes a plurality of information elements and relationship information indicating relationship among the plurality of information elements. The information extracting unit extracts the response addition information from a group of information elements having relationship with an information element extracted as the response basic information and an information element included in the knowledge information relating to the user, directly or via one or more information elements.

Claims

1. An information processing apparatus, comprising:

at least one sensor configured to acquire environmental information;

a microphone configured to acquire a first sound signal of a user; and

a Central Processing Unit (CPU) configured to;

store, in a storage apparatus, knowledge information comprising a plurality of information elements, relationship information, and the acquired environmental information,

wherein the plurality of information elements include information relating to the user, and

wherein the relationship information indicates relationship among the plurality of information elements;

convert, based on sound recognition, the acquired first sound signal into text information;

detect a question corresponding to the first sound signal based on the text information;

detect, based on morphological analysis, intention of the detected question;

generate a graphical structure that indicates linkage among a first group of information elements, based on the detected intention of the question,

wherein each of a plurality of nodes of the graphical structure corresponds to one of the first group of information elements, and

wherein corresponding score associated with each of arcs between at least two of the plurality of nodes is set in advance;

extract response basic information which is a response to the detected question and response addition information, based on the generated graphical structure and the acquired environmental information,

wherein the response addition information has relationship with the response basic information based on the stored knowledge information,

wherein the response addition information is extracted from a second group of information elements included in the first group of information elements having relationship with a first information element extracted as the response basic information and a second information element included in the knowledge information relating to the user, based on at least one information element from the plurality of information elements,

wherein the at least one information element from the second group of information elements is selected based on strength of relationship between each of the plurality of information elements and the second information element included in the knowledge information relating to the user, and

wherein the second group of information elements lacks relationship with a third information element corresponding to the question;

output, based on sound synthesis, the extracted response basic information and the extracted response addition information as a second sound signal; and

update the corresponding score based on a user feedback on the outputted second sound signal.

2. The information processing apparatus according to claim 1, wherein the CPU is further configured to:

extract a fourth information element that is directly associated with a particular information element; and

extract a fifth information element as the response addition information,

wherein the fifth information element having a particular relationship with the extracted fourth information element.

3. The information processing apparatus according to claim 2, wherein the particular relationship means that the fourth information element and the fifth information element have relationship of an event and a countermeasure to the event.

4. The information processing apparatus according to claim 3,

wherein based on the second group of information elements that includes a plurality of fourth information elements, the CPU is further configured to:

select at least one of the plurality of fourth information elements from the plurality of fourth information elements based on scores that are set in advance for each of the plurality of fourth information elements; and

extract the fifth information element having the particular relationship with the selected at least one of the plurality of fourth information elements as the response addition information.

5. The information processing apparatus according to claim 3, wherein based on the second group of the information elements that includes a plurality of fourth information elements, the CPU is further configured to:

select at least one of the plurality of fourth information elements from the plurality of fourth information elements based on the strength of relationship between each of the plurality of fourth information elements and the second information element included in the knowledge information relating to the user; and

extract the fifth information element having the particular relationship with the selected at least one of the plurality of fourth information elements as the response addition information.

6. The information processing apparatus according to claim 1, wherein the CPU is further configured to extract, as the response addition information, a sixth information element associated with a particular information element.

7. The information processing apparatus according to claim 1, the CPU is further configured to generate a response sentence based on the response basic information and the response addition information.

8. The information processing apparatus according to claim 1, wherein the knowledge information further includes at least one of common knowledge information or dynamic information indicating an event in the world.

9. An information processing method, comprising:

acquiring environmental information from at least one sensor;

acquiring, from a microphone, a first sound signal of a user;

storing, in a storage apparatus, knowledge information comprising a plurality of information elements, relationship information, and the acquired environmental information,

wherein the plurality of information elements include information relating to the user, and

wherein the relationship information indicates relationship among the plurality of information elements;

converting, based on sound recognition, the acquired first sound signal into text information;

detecting a question corresponding to the first sound signal based on the text information;

detecting, based on morphological analysis, intention of the detected question;

generating a graphical structure that indicates linkage among a first group of information elements, based on the detected intention of the question,

wherein each of a plurality of nodes of the graphical structure corresponds to one of the first group of information elements, and

wherein corresponding score associated with each of arcs between at least two of the plurality of nodes is set in advance;

extracting response basic information which is a response to the detected question and response addition information, based on the generated graphical structure and the acquired environmental information,

wherein the response addition information has relationship with the response basic information based on the stored knowledge information,

wherein the response addition information is extracted from a second group of information elements included in the first group of information elements having relationship with a first information element extracted as the response basic information and a second information element included in the knowledge information relating to the user, based on at least one information element from the plurality of information elements,

wherein the at least one information element from the second group of information elements is selected based on strength of relationship between each of the plurality of information elements and the second information element included in the knowledge information relating to the user, and

wherein the second group of information elements lacks relationship with a third information element corresponding to the question;

outputting, based on sound synthesis, the extracted response basic information and the extracted response addition information as a second sound signal; and

updating the corresponding score based on a user feedback on the outputted second sound signal.

10. A non-transitory computer-readable medium having stored thereon computer-readable instructions, which when executed by a computer, cause the computer to execute operations, the operations comprising:

acquiring environmental information from at least one sensor;

acquiring, from a microphone, a first sound signal of a user,

storing, in a storage apparatus, knowledge information comprising a plurality of information elements, relationship information, and the acquired environmental information,

wherein the plurality of information elements include information relating to the user, and

wherein the relationship information indicates relationship among the plurality of information elements;

converting, based on sound recognition, the acquired first sound signal into text information;

detecting a question corresponding to the first sound signal based on the text information;

detecting, based on morphological analysis, intention of the detected question;

generating a graphical structure that indicates linkage among a first group of information elements, based on the detected intention of the question,

wherein each of a plurality of nodes of the graphical structure corresponds to one of the first group of information elements, and

wherein corresponding score associated with each of arcs between at least two of the plurality of nodes is set in advance;

extracting response basic information which is a response to the detected question and response addition information, based on the generated graphical structure and the acquired environmental information,

wherein the response basic information has relationship with the response basic information based on the stored knowledge information,

wherein the response addition information is extracted from a second group of information elements included in the first group of information elements having relationship with a first information element extracted as the response basic information and a second information element included in the knowledge information relating to the user, based on at least one information element from the plurality of information elements,

wherein the at least one information element from the second group of information elements is selected based on strength of relationship between each of the plurality of information elements and the second information element included in the knowledge information relating to the user, and

wherein the second group of information elements lacks relationship with a third information element corresponding to the question; and

controlling, based on sound synthesis, output of the extracted response basic information and the extracted response addition information as a second sound signal; and

updating the corresponding score based on a user feedback on the outputted second sound signal.