Information processing apparatus, information processing method, and program

• US 9,477,304 B2
• Filed: 05/26/2011
• Issued: 10/25/2016
• Est. Priority Date: 06/02/2010
• Status: Active Grant

• Current Assignee: Sony Corporation (Sony Group Corp.)
• Original Assignee: Sony Corporation (Sony Group Corp.)
• Inventors: Sano, Akane
• Primary Examiner(s): BAKER, MATTHEW H
• Application Number:
  US13/116,848
• Publication Number:
  US 20110301956A1
• Time in Patent Office: 1,979 Days
• Field of Search: 704/270, 704/270.1
• US Class Current: 1/1
• CPC Class Codes: G06F 3/012 Head tracking input arrangements
• CPC Class Codes: G06F 3/16 Sound input; Sound output speech processing G10L
• CPC Class Codes: G06V 40/16 Human faces, e.g. facial parts, sketches or expressions
• CPC Class Codes: G10L 15/26 Speech to text systems G10L15/08 takes precedence

Abstract

An information processing apparatus includes an image analysis unit that executes a process for analyzing an image captured by a camera, a speech analysis unit that executes a process for analyzing speech input from a microphone, and a data processing unit that receives a result of the analysis conducted by the image analysis unit and a result of the analysis conducted by the speech analysis unit and that executes output control of help information for a user. The data processing unit calculates a degree of difficulty of the user on the basis of at least either the result of the image analysis or the result of the speech analysis and, if the degree of difficulty that has been calculated is equal to or more than a predetermined threshold value, executes a process for outputting help information to the user.

Claims

1. An information processing electronic device configured to:

execute a first process for analyzing an image captured by a camera and for determining whether the image contains a face;

execute a second process for analyzing speech input from a microphone;

receive a result of the first process and a result of the second process and execute output control of help information for a user;

judge a duration over which a face of the user faces a particular direction on the basis of the result of the first process, if the image is determined to contain a face;

determine a number of times the information processing electronic device rejects execution of a process, corresponding to the speech input, in response to the speech input made by the user;

determine a user level of the user based on the number of times the information processing electronic device rejects the execution of the process corresponding to the speech input, a feature quantity and a predetermined threshold value for the number of times of rejecting the execution of the process, wherein the feature quantity represents information regarding reasons for rejecting the execution of the process; and

calculate a degree of difficulty of the user by using information regarding the judgment, the number of times and the determined user level, and if the degree of difficulty that has been calculated is equal to or more than a predetermined threshold value, execute a process for outputting the help information to the user.

2. The information processing electronic device according to claim 1, further configured to:

determine whether or not a face of the user faces the information processing electronic device on the basis of the result of the first process and calculate the degree of difficulty by using information regarding the determination.

3. The information processing electronic device according to claim 1, further configured to:

calculate the degree of difficulty by using information regarding a judgment as to whether or not a process corresponding to a request made by the user has been executed.

4. The information processing electronic device according to claim 1, further configured to:

calculate the degree of difficulty on the basis of information regarding a time elapsed since the information processing electronic device executed a process for responding to the user.

5. The information processing electronic device according to claim 1, further configured to:

obtain and store state transition of the information processing electronic device; and

execute a process for outputting help information corresponding to a stored system state.

6. The information processing electronic device of claim 1, wherein the first process comprises the information processing electronic device being configured to:

responsive to determining the image contains a face;

determine whether the face contained in the image is a face of the user;

estimate an angle of the face with respect to the information processing electronic device; and

responsive to determining that the face contained in the image is a face of the user, determine that the face of the user faces the particular direction based on the estimated angle.

7. The information processing electronic device according to claim 1,

wherein the number of times the information processing electronic device rejects the execution of the process corresponding to the speech input is determined for a session, and

wherein the session refers to a period of time until the information processing electronic device executes the process corresponding to the speech input.

8. The information procession electronic device according to claim 1, wherein the reason for rejecting the execution of the process includes at least one of:

a failure in detection of voiced frames, a judgment that an utterance has been made outside a domain, and results of speech analysis judged to have low reliability.

9. The information processing electronic device according to claim 1, further configured to determine a learning level of the user in a predetermined period of time until a present time.

10. The information processing electronic device according to claim 1, further configured to match a direction of the face of the user at a first instance with the direction of the face of the user at a second instance, wherein the second instance is before the first instance.

11. The information processing electronic device according to claim 1, further configured to compare the duration over which the face of the user faces the particular direction with a predetermined threshold time to determine the degree of difficulty of the user.

12. An information processing method that is used in an information processing apparatus, the information processing method comprising:

executing, with an image analysis unit, a process for analyzing an image captured by a camera;

executing, with a speech analysis unit, a process for analyzing speech input from a microphone; and

receiving, with a data processing unit, a result of the analysis conducted by the image analysis unit and a result of the analysis conducted by the speech analysis unit and executing output control of help information for a user,

wherein, in the receiving, a degree of difficulty of the user is calculated based on;

a duration over which a face of the user faces a particular direction, if the image is determined to contain a face, the duration being determined based on the result of the image analysis,

a number of times the information processing apparatus rejects execution of a process, corresponding to the speech input, in response to the speech input made by the user, and

a user level of the user determined based on the number of times the information processing electronic device rejects the execution of the process corresponding to the speech input, a feature quantity and a predetermined threshold value for the number of times of rejecting the execution of the process, wherein the feature quantity represents information regarding reasons for rejecting the execution of the process, and

if the degree of difficulty that has been calculated is equal to or more than a predetermined threshold value, a process for outputting the help information to the user is executed.

13. A non-transitory computer-readable medium comprising instructions that, when executed by an information processing apparatus, cause the information processing apparatus to:

cause an image analysis unit to execute a process for analyzing an image captured by a camera;

cause a speech analysis unit to execute a process for analyzing speech input from a microphone; and

cause a data processing unit to receive a result of the analysis conducted by the image analysis unit and a result of the analysis conducted by the speech analysis unit and to execute output control of help information to a user,

wherein, in the causing of the data processing unit to receive the result, a degree of difficulty of the user is calculated based on;

a duration over which a face of the user faces a particular direction, if the image is determined to contain a face, the duration being determined based on the result of the image analysis,

a number of times the information processing apparatus rejects execution of a process, corresponding to the speech input, in response to the speech input made by the user, and

a user level of the user determined based on the number of times the information processing electronic device rejects the execution of the process corresponding to the speech input, a feature quantity and a predetermined threshold value for the number of times of rejecting the execution of the process, wherein the feature quantity represents information regarding reasons for rejecting the execution of the process, and

if the degree of difficulty that has been calculated is equal to or more than a predetermined threshold value, a process for outputting the help information to the user is executed.