Embedded Conversational Agent-Based Kiosk for Automated Interviewing

• US 20130266925A1
• Filed: 01/30/2013
• Published: 10/10/2013
• Est. Priority Date: 01/30/2012
• Status: Abandoned Application

• Current Assignee: Arizona Board of Regents (University of Arizona)
• Original Assignee: Arizona Board of Regents (University of Arizona)
• Inventors: Nunamaker, Jay F. Jr., Burgoon, Judee K., Elkins, Aaron C., Patton, Mark W., Derrick, Douglas C., Moffitt, Kevin C.
• Application Number:
  US13/754,557
• Publication Number:
  US 20130266925A1
• Time in Patent Office: Days
• US Class Current: 434/362
• CPC Class Codes: G06N 20/00 Machine learning
• CPC Class Codes: G09B 7/00 Electrically-operated teaching apparatus or devices working with questions and answers mechanically operated G09B3/00; computing arrangements G06F

Abstract

Methods and systems for interviewing human subjects are disclosed. A user interface of a computing device can direct a question to a human subject. The computing device can receive a response from the human subject related to the question. The response can be received using one or more sensors associated with the computing device. The computing device can generate a classification of the response. The computing device can determine a next question based on a script tree and the classification. The computing device can direct the next question to the human subject using the user interface of the computing device.

Claims

1. A system, comprising:

a processor;

a user interface;

one or more sensors;

a non-transitory computer readable medium configured to store at least a script tree and program instructions that, upon execution by the processor, cause the system to perform operations comprising;

directing a question to a human subject using the user interface;

receiving a response from the human subject related to the question using the one or more sensors;

generating a classification of the response;

determining a next question based on the script tree and the classification; and

directing the next question to the human subject using the user interface.

2. The system of claim 1, wherein the operation of directing the question to the human subject comprises communicating the question to the human subject using an intelligent agent and the user interface, wherein the intelligent agent comprises an animation of a human face.

3. The system of claim 2, wherein the operation of receiving the response related to the question comprises displaying a gesture using the intelligent agent, wherein the gesture is based on the question.

4. The system of claim 3, wherein the gesture is at least one gesture selected from the group consisting of a head nod gesture, a head shake gesture, and a smile gesture.

5. The system of claim 2, wherein the intelligent agent is configured to communicate with the human subject using generated speech based upon one or more voice parameters.

6. The system of claim 5, wherein the one or more voice parameters comprise at least one voice parameter selected from the group consisting of a gender voice parameter, a pitch voice parameter, a tempo voice parameter, a volume voice parameter, and an accent voice parameter.

7. The system of claim 2, wherein the one or more sensors are configured to be controlled by the intelligent agent.

8. The system of claim 2, wherein the human face is selected to be either a human face representing a male agent or a human face representing a female agent.

9. The system of claim 2, wherein the human face is selected to be either a human face representing a neutral agent or a human face representing a smiling agent.

10. The system of claim 1, wherein the response comprises speech from the human agent, and wherein the operation of generating the classification of the response comprises analyzing the speech from the human subject.

11. The system of claim 10, wherein analyzing the speech from the human subject comprises:

determining an average pitch of the speech from the human subject;

determining a pitch and a harmonics-to-noise ratio (HNR) of a portion of the speech from the human subject, wherein the HNR determines a voice quality;

determining whether the pitch is above the average pitch and determining whether the HNR increases in the portion of the speech from the human subject;

in response to determining that the pitch is below the average pitch and that the HNR increases, classifying the response as a response to a stressful question; and

in response to determining that the pitch is above the average pitch and that the HNR increases, classifying the response as a potentially untruthful response.

12. The system of claim 10, wherein analyzing the speech from the human subject comprises:

determining a value based on the pitch of the speech from the human subject;

determining whether the value based on the pitch is above a threshold value; and

in response to determining that the value based on the pitch is above the threshold value, classifying the response as a response to a stressful question.

13. The system of claim 1, further comprising an operator interface to the user interface, wherein the operator interface is configured to provide information about the question, the response, and the classification.

14. A method, comprising:

directing a question to a human subject using a user interface to a computing device;

receiving a response from the human subject related to the question using one or more sensors associated with the computing device;

generating a classification of the response using the computing device;

determining a next question based on a script tree and the classification using the computing device; and

directing the next question to the human subject using the user interface of the computing device.

15. The method of claim 14, wherein directing the question to the human subject comprises communicating the question to the human subject using an intelligent agent of the computing device, wherein the intelligent agent comprises an animation of a human face.

16. The method of claim 15, wherein receiving the response related to the question comprises displaying a gesture using the intelligent agent, wherein the gesture is based on the question.

17. The method of claim 16, wherein the gesture is at least one gesture selected from the group consisting of a head nod gesture, a head shake gesture, and a smile gesture.

18. The method of claim 15, wherein the intelligent agent is configured to communicate with the human subject using generated speech based upon one or more voice parameters.

19. The method of claim 18, wherein the one or more voice parameters comprise at least one voice parameter selected from the group consisting of a gender voice parameter, a pitch voice parameter, a tempo voice parameter, a volume voice parameter, and an accent voice parameter.

20. The method of claim 15, wherein the one or more sensors are configured to be controlled by the intelligent agent.

21. The method of claim 15, wherein the human face is selected to be either a human face representing a male agent or a human face representing a female agent.

22. The method of claim 15, wherein the human face is selected to be either a human face representing a neutral agent or a human face representing a smiling agent.

23. The method of claim 14, wherein the response comprises speech from the human subject, and wherein generating the classification of the response comprises analyzing the speech from the human subject.

24. The method of claim 23, wherein analyzing the speech from the human subject comprises:

determining an average pitch of the speech from the human subject;

determining a pitch and a harmonics-to-noise ratio (HNR) of a portion of the speech from the human subject, wherein the HNR determines a voice quality;

determining whether the pitch is above the average pitch and determining whether the HNR increases in the portion of the speech from the human subject;

in response to determining that the pitch is below the average pitch and that the HNR increases, classifying the response as a response to a stressful question; and

in response to determining that the pitch is above the average pitch and that the HNR increases, classifying the response as a potentially untruthful response.

25. The method of claim 23, wherein analyzing the speech from the human subject comprises:

determining a value based on a pitch of the speech from the human subject;

determining whether the value based on the pitch is above a threshold value; and

in response to determining that the value based on the pitch is above the threshold value, classifying the response as a response to a stressful question.

26. The method of claim 14, further comprising:

providing information about the question, the response, and the classification via an operator interface to the user interface.

27. A non-transitory computer-readable storage medium having stored thereon program instructions that, upon execution by a computing device, cause the computing device to perform operations comprising:

directing a question to a human subject using a user interface to a computing device;

receiving a response from the human subject related to the question using one or more sensors associated with the computing device;

generating a classification of the response using the computing device;

determining a next question based on a script tree and the classification using the computing device; and

directing the next question to the human subject using the user interface of the computing device.

28. The non-transitory computer-readable storage medium of claim 27, wherein the operation of directing the question to the human subject comprises communicating the question to the human subject using an intelligent agent of the computing device, wherein the intelligent agent comprises an animation of a human face.

29. The non-transitory computer-readable storage medium of claim 28, wherein the operation of receiving the response related to the question comprises displaying a gesture using the intelligent agent, wherein the gesture is based on the question.

30. The non-transitory computer-readable storage medium of claim 29, wherein the gesture is at least one gesture selected from the group consisting of a head nod gesture, a head shake gesture, and a smile gesture.

31. The non-transitory computer-readable storage medium of claim 28, wherein the intelligent agent is configured to communicate with the human subject using generated speech based upon one or more voice parameters.

32. The non-transitory computer-readable storage medium of claim 31, wherein the one or more voice parameters comprise at least one voice parameter selected from the group consisting of a gender voice parameter, a pitch voice parameter, a tempo voice parameter, a volume voice parameter, and an accent voice parameter.

33. The non-transitory computer-readable storage medium of claim 28, wherein the one or more sensors are configured to be controlled by the intelligent agent.

34. The non-transitory computer-readable storage medium of claim 28 wherein the human face is selected to be either a human face representing a male agent or a human face representing a female agent.

35. The non-transitory computer-readable storage medium of claim 28, wherein the human face is selected to be either a human face representing a neutral agent or a human face representing a smiling agent.

36. The non-transitory computer-readable storage medium of claim 27, wherein the response comprises speech from the human agent, and wherein the operation of generating the classification of the response comprises analyzing the speech from the human subject.

37. The non-transitory computer-readable storage medium of claim 36, wherein analyzing the speech from the human subject comprises:

determining an average pitch of the speech from the human subject;

determining a pitch and a harmonics-to-noise ratio (HNR) of a portion of the speech from the human subject, wherein the HNR determines a voice quality;

determining whether the pitch is above the average pitch and determining whether the HNR increases in the portion of the speech from the human subject;

in response to determining that the pitch is below the average pitch and that the HNR increases, classifying the response as a response to a stressful question; and

in response to determining that the pitch is above the average pitch and that the HNR increases, classifying the response as a potentially untruthful response.

38. The non-transitory computer-readable storage medium of claim 36, wherein analyzing the speech from the human subject comprises:

determining a value based on the pitch of the speech from the human subject;

determining whether the value based on the pitch is above a threshold value; and

in response to determining that the value based on the pitch is above the threshold value, classifying the response as a response to a stressful question.

39. The non-transitory computer-readable storage medium of claim 27, wherein the functions further comprise:

providing information about the question, the response, and the classification via an operator interface to the user interface.