Information processing apparatus, information processing method, and program for providing feedback on a user request

• US 8,983,846 B2
• Filed: 03/22/2011
• Issued: 03/17/2015
• Est. Priority Date: 03/29/2010
• Status: Expired due to Fees

• Current Assignee: Sony Corporation (Sony Group Corp.)
• Original Assignee: Sony Corporation (Sony Group Corp.)
• Inventors: Di Profio, Ugo, Sano, Akane, Matsuda, Kouichi, Sawada, Tsutomu, Yamada, Keiichi
• Primary Examiner(s): YEN, ERIC L
• Application Number:
  US13/069,237
• Publication Number:
  US 20110282673A1
• Time in Patent Office: 1,456 Days
• Field of Search: 704270-275
• US Class Current: 704/275
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: H04N 21/42203 sound input device, e.g. microphone
• CPC Class Codes: H04N 21/42204 User interfaces specially adapted for controlling a client device through a remote control device; Remote control devices therefor constructive details of casings for the remote control device H01H9/0235; user interfaces for controlling a tuning device of a television receiver through a remote control H03J9/00; remote control of peripheral devices connected to a television receiver through the remote control device of the television receiver H04B1/205
• CPC Class Codes: H04N 21/4223 Cameras H04N23/00 takes precedence
• CPC Class Codes: H04N 21/44008 involving operations for analysing video streams, e.g. detecting features or characteristics in the video stream arrangements characterised by components specially adapted for monitoring, identification or recognition of video in broadcast systems H04H60/59
• CPC Class Codes: H04N 21/8146 involving graphical data, e.g. 3D object, 2D graphics

Abstract

Provided is an information processing apparatus including: a voice analysis unit which performs an analysis process for a user speech; and a data processing unit which is input with analysis results of the voice analysis unit to determine a process which is to be performed by the information processing apparatus, wherein in the case where a factor of inhibiting process continuation occurs in a process based on the user speech, the data processing unit performs a process of generating and outputting feedback information corresponding to a process stage in which the factor of inhibiting occurs.

Claims

1. An information processing apparatus comprising:

a voice analysis unit configured to perform an analysis process for a user speech, the user speech including an instruction for performing a desired operation; and

a data processing unit configured to receive analysis results of the voice analysis unit to determine a process which is to be performed by the information processing apparatus, the process corresponding to the instruction given by the user through the user speech,

wherein in a case where a factor of inhibiting process continuation occurs based on at least one of the voice analysis unit and the data processing unit determining that the instruction given by the user is not understood, the data processing unit performs a process of generating and outputting feedback information to the user, in which the factor of inhibiting process continuation occurs as a result of a failure to understand the instruction given by the user,

wherein the data processing unit is further configured to perform a process of displaying an avatar as a status indicator of the information processing apparatus as the outputted feedback information,

wherein the avatar visually depicts a status of the information processing apparatus, as a feedback notification to the user, by visually indicating that the user speech is being received or analyzed at a time when the user speech is being received or analyzed, by visually indicating that the instruction included in the user speech is not understood when the instruction included in the user speech is not understood, by visually indicating that the instruction included in the user speech is understood when the instruction included in the user speech is understood, by visually indicating that the process which is to be performed in correspondence with the instruction given by the user cannot successfully be determined when the data processing unit cannot successfully determine the process which is to be performed in correspondence with the instruction given by the user, and when the instruction for performing the desired operation has been understood, the information processing apparatus performs the process corresponding to the instruction and the avatar is changed to visually indicate that the desired operation is being performed, and

wherein at least one of the voice analysis unit and the data processing unit is implemented via a processor.

2. The information processing apparatus according to claim 1,

wherein the information processing apparatus further comprises an image analysis unit configured to analyze a user action, and

wherein the data processing unit is further configured to receive analysis results of the image analysis unit to determine the process which is to be performed by the information processing apparatus, the process corresponding to an instruction given by the user through the user action, and in a case where a second factor of inhibiting process continuation occurs based on at least one of the image analysis unit and the data processing unit determining that the instruction given by the user through the user action is not understood, the data processing unit performs a process of generating and outputting feedback information to the user, in which the second factor of inhibiting process continuation occurs as a result of a failure to understand the instruction given by the user,

wherein at least one of the voice analysis unit, the data processing unit, and the image analysis unit is implemented via a processor.

3. The information processing apparatus according to claim 1, wherein the data processing unit changes an appearance of the displayed avatar based on whether or not the instruction given by the user is understood.

4. The information processing apparatus according to claim 1, wherein the data processing unit changes a facial expression of the displayed avatar based on whether or not the instruction given by the user is understood.

5. The information processing apparatus according to claim 4, wherein the facial expression of the displayed avatar indicates a current mood based on whether or not the instruction given by the user is understood.

6. The information processing apparatus according to claim 1,

wherein the avatar is a static image that conveys a visual notification as to whether or not the instruction given by the user is understood.

7. The information processing apparatus according to claim 1,

wherein the avatar depicts a facial expression that visually conveys a state of emotion corresponding to whether or not the instruction given by the user is understood.

8. The information processing apparatus according to claim 1,

wherein the avatar visually expresses a state of emotion, the state of emotion based upon whether or not the instruction given by the user is understood.

9. An information processing method of performing an information process in an information processing apparatus, comprising the steps of:

performing an analysis process for a user speech in an voice analysis unit, the user speech including an instruction for performing a desired operation; and

receiving analysis results of the performing of the analysis process to determine a process which is to be performed by the information processing apparatus, the process corresponding to the instruction given by the user through the user speech,

wherein when a factor of inhibiting process continuation occurs based on a determination that the instruction given by the user is not understood during at least one of the performing of the analysis process and the determining of the process which is to be performed by the information processing apparatus, a feedback information is generated and outputted to the user,

wherein the factor of inhibiting process continuation occurs as a result of a failure to understand the instruction given by the user,

wherein an avatar is displayed as a status indicator of the information processing apparatus as the outputted feedback information,

wherein the avatar visually depicts a status of the information processing apparatus, as a feedback notification to the user, by visually indicating that the user speech is being received or analyzed at a time when the user speech is being received or analyzed, by visually indicating that the instruction included in the user speech is not understood when the instruction included in the user speech is not understood, by visually indicating that the instruction included in the user speech is understood when the instruction included in the user speech is understood, by visually indicating that the process which is to be performed in correspondence with the instruction given by the user cannot successfully be determined when the process which is to be performed in correspondence with the instruction given by the user cannot successfully be determined, and when the instruction for performing the desired operation has been understood, the process corresponding to the instruction is performed and the avatar is changed to visually indicate that the desired operation is being performed, and

wherein the inputting is implemented via a processor.

10. The information processing method according to claim 9, wherein an appearance of the displayed avatar is changed based on whether or not the instruction given by the user is understood.

11. The information processing method according to claim 9, wherein a facial expression of the displayed avatar is changed based on whether or not the instruction given by the user is understood.

12. A non-transitory computer-readable medium having embodied thereon a program, which when executed by a processor in an information processing apparatus causes the information processing apparatus to execute a method, the method comprising:

allowing a voice analysis unit to perform an analysis process for a user speech, the user speech including an instruction for performing a desired operation; and

allowing a data processing unit to be inputted with analysis results of the performing of the analysis process for the user speech, and determining a process which is to be performed by the information processing apparatus, the process corresponding to the instruction given by the user through the user speech,

wherein when a factor of inhibiting process continuation occurs based on a determination that the instruction given by the user is not understood during at least one of the performing of the analysis process and the determining of the process which is to be performed by the information processing apparatus, a feedback information is generated and outputted to the user,

wherein the factor of inhibiting process continuation occurs as a result of a failure to understand the instruction given by the user,

wherein an avatar is displayed as a status indicator of the information processing apparatus as the outputted feedback information,

wherein the avatar visually depicts a status of the information processing apparatus, as a feedback notification to the user, by visually indicating that the user speech is being received or analyzed at a time when the user speech is being received or analyzed, by visually indicating that the instruction included in the user speech is not understood when the instruction included in the user speech is not understood, by visually indicating that the instruction included in the user speech is understood when the instruction included in the user speech is understood, by visually indicating that the process which is to be performed in correspondence with the instruction given by the user cannot successfully be determined when the process which is to be performed in correspondence with the instruction given by the user cannot successfully be determined, and when the instruction for performing the desired operation has been understood, the process corresponding to the instruction is performed and the avatar is changed to visually indicate that the desired operation is being performed.

13. The non-transitory computer-readable medium according to claim 12, wherein in the executed method, an appearance of the displayed avatar is changed based on whether or not the instruction given by the user is understood.

14. The non-transitory computer-readable medium according to claim 12, wherein in the executed method, a facial expression of the displayed avatar is changed based on whether or not the instruction given by the user is understood.