INTERNET OF THINGS PUBLIC SPEAKING COACH

• US 20200135050A1
• Filed: 10/31/2018
• Published: 04/30/2020
• Est. Priority Date: 10/31/2018
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Monge Nunez, Roxana
• Granted Patent: US 11,817,005 B2
• Time in Patent Office: Days
• CPC Class Codes: G06N 20/00 Machine learning
• CPC Class Codes: G09B 19/04 Speaking with audible presentation of the material to be studied G09B5/04
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 15/26 Speech to text systems G10L15/08 takes precedence
• CPC Class Codes: G10L 25/48 specially adapted for particular use
• CPC Class Codes: G10L 25/60 for measuring the quality of voice signals
• CPC Class Codes: G10L 25/63 for estimating an emotional state
• CPC Class Codes: G10L 25/90 Pitch determination of speech signals

Abstract

Approaches presented herein enable delivery of real-time internet of things (IoT) feedback to optimize a public speaking performance. More specifically, a set of data representing a speaking performance of a user is captured and analyzed to generate a speaking performance profile of the user. This profile is compared to a reference speaking performance profile and, based on the comparison, a set of performance improvement strategies for the user is generated. A performance improvement strategy is selected from the set of performance improvement strategies based on an identification of an availability of a set of IoT devices for delivery of at least one of the strategies. Instructions are then communicated, responsive to the captured speaking performance associated with the user, to an available IoT device to deliver the selected performance improvement strategy to the user through an output user interface of the available IoT device during the speaking performance.

Claims

1. A method for improving a spoken presentation of a user in real time using internet of things (IoT) feedback, the method comprising:

capturing a set of data representative of a speaking performance associated with a user;

analyzing the captured data by assigning a set of values to a verbal element of the speaking performance of the user, a non-verbal element of the speaking performance of the user, and a paralanguage element of the speaking performance of the user, to generate a speaking performance profile of the user;

comparing the speaking performance profile of the user to a reference speaking performance profile having a set of values that indicate an optimal verbal speaking performance, non-verbal speaking performance, and paralanguage speaking performance for the user;

generating, based on the comparison, a set of performance improvement strategies for the user;

selecting a performance improvement strategy from the set of performance improvement strategies based on an identification of an availability of a set of IoT devices for delivery of at least one of the set of performance improvement strategies; and

communicating, responsive to the captured speaking performance associated with the user, instructions to deliver the selected performance improvement strategy to the user through an output user interface of the available IoT device during the speaking performance.

2. The method of claim 1, the method further comprising:

detecting a response of the user to the delivery of the selected performance improvement strategy through the output user interface of the available IoT device during the speaking performance; and

adding the detected user response to the selected performance improvement strategy to the speaking performance profile of the user.

3. The method of claim 2, the method further comprising:

ranking the available set of performance improvement strategies for the user according to a historic effectiveness of each of the set of performance improvement strategies; and

communicating, responsive to the detected response of the user to the delivery of the selected performance improvement strategy being a negative response, instructions to deliver a next best performance improvement strategy according to the ranking to one of the available IoT devices during the speaking performance.

4. The method of claim 1, the method further comprising:

detecting an audience reaction to the speaking performance associated with the user; and

generating the speaking performance profile of the user based on the detected audience reaction.

5. The method of claim 1, wherein the set of data comprises video, audio, or sensor data,

wherein the verbal element of the speaking performance comprises a verbal element selected from the group consisting of:

a spoken word, and a spoken word meaning;

wherein the non-verbal element of the speaking performance comprises a non-verbal element selected from the group consisting of;

posture, mood, use of space, and gestures, and

wherein the paralanguage element of the speaking performance comprises a paralanguage element selected from the group consisting of;

voice quality, voice rate, voice pitch, voice volume, mood, voice rhythm, voice intonation, stress, and speaking style.

6. The method of claim 1, wherein at least one of the set of performance improvement strategies comprises a prompt, performable by an IoT device, for the user to perform a designated action, and wherein at least one of the set of IoT devices is a device selected from the group consisting of:

an audio-playing IoT device, a light display IoT device, a haptic IoT device, and a display screen IoT device.

7. The method of claim 1, the method further comprising:

generating a public speaking performance scenario having a set of parameter constraints;

communicating the public speaking performance scenario to the user; and

monitoring the public speaking performance of the user under the set of parameter constraints.

8. The method of claim 1, the method further comprising using a machine learning algorithm to generate the speaking performance profile of the user based on previous speaking performance interactions of the user with IoT devices delivering performance improvement strategies.

9. A computer system for improving a spoken presentation of a user in real time using internet of things (IoT) feedback, the computer system comprising:

a memory medium comprising program instructions;

a bus coupled to the memory medium; and

a processor, for executing the program instructions, coupled to an IoT public speaking coach engine via the bus that when executing the program instructions causes the system to;

capture a set of data representative of a speaking performance associated with a user;

analyze the captured data by assigning a set of values to a verbal element of the speaking performance of the user, a non-verbal element of the speaking performance of the user, and a paralanguage element of the speaking performance of the user, to generate a speaking performance profile of the user;

compare the speaking performance profile of the user to a reference speaking performance profile having a set of values that indicate an optimal verbal speaking performance, non-verbal speaking performance, and paralanguage speaking performance for the user;

generate, based on the comparison, a set of performance improvement strategies for the user;

select a performance improvement strategy from the set of performance improvement strategies based on an identification of an availability of a set of IoT devices for delivery of at least one of the set of performance improvement strategies; and

communicate, responsive to the captured speaking performance associated with the user, instructions to deliver the selected performance improvement strategy to the user through an output user interface of the available IoT device during the speaking performance.

10. The computer system of claim 9, the instructions further causing the system to:

detect a response of the user to the delivery of the selected performance improvement strategy through the output user interface of the available IoT device during the speaking performance; and

add the detected user response to the selected performance improvement strategy to the speaking performance profile of the user.

11. The computer system of claim 10, the instructions further causing the system to:

rank the available set of performance improvement strategies for the user according to a historic effectiveness of each of the set of performance improvement strategies; and

communicate, responsive to the detected response of the user to the delivery of the selected performance improvement strategy being a negative response, instructions to deliver a next best performance improvement strategy according to the ranking to one of the available IoT devices during the speaking performance.

12. The computer system of claim 9, the instructions further causing the system to:

detect an audience reaction to the speaking performance associated with the user; and

generate the speaking performance profile of the user based on the detected audience reaction.

13. The computer system of claim 9, wherein the set of data comprises video, audio, or sensor data,

wherein the verbal element of the speaking performance comprises a verbal element selected from the group consisting of:

a spoken word, and a spoken word meaning;

wherein the non-verbal element of the speaking performance comprises a non-verbal element selected from the group consisting of;

posture, mood, use of space, and gestures, and

wherein the paralanguage element of the speaking performance comprises a paralanguage element selected from the group consisting of;

voice quality, voice rate, voice pitch, voice volume, mood, voice rhythm, voice intonation, stress, and speaking style.

14. The computer system of claim 9, wherein at least one of the set of performance improvement strategies comprises a prompt, performable by an IoT device, for the user to perform a designated action, and wherein at least one of the set of IoT devices is a device selected from the group consisting of:

an audio-playing IoT device, a light display IoT device, a haptic IoT device, and a display screen IoT device.

15. The computer system of claim 9, the instructions further causing the system to:

generate a public speaking performance scenario having a set of parameter constraints;

communicate the public speaking performance scenario to the user; and

monitor the public speaking performance of the user under the set of parameter constraints.

16. The computer system of claim 9, the instructions further causing the system to use a machine learning algorithm to generate the speaking performance profile of the user based on previous speaking performance interactions of the user with IoT devices delivering performance improvement strategies.

17. A computer program product for improving a spoken presentation of a user in real time using internet of things (IoT) feedback, the computer program product comprising a computer readable hardware storage device, and program instructions stored on the computer readable hardware storage device, to:

capture a set of data representative of a speaking performance associated with a user;

analyze the captured data by assigning a set of values to a verbal element of the speaking performance of the user, a non-verbal element of the speaking performance of the user, and a paralanguage element of the speaking performance of the user, to generate a speaking performance profile of the user;

compare the speaking performance profile of the user to a reference speaking performance profile having a set of values that indicate an optimal verbal speaking performance, non-verbal speaking performance, and paralanguage speaking performance for the user;

generate, based on the comparison, a set of performance improvement strategies for the user;

select a performance improvement strategy from the set of performance improvement strategies based on an identification of an availability of a set of IoT devices for delivery of at least one of the set of performance improvement strategies; and

communicate, responsive to the captured speaking performance associated with the user, instructions to deliver the selected performance improvement strategy to the user through an output user interface of the available IoT device during the speaking performance.

18. The computer program product of claim 17, the computer readable storage device further comprising instructions to:

rank the available set of performance improvement strategies for the user according to a historic effectiveness of each of the set of performance improvement strategies;

detect a response of the user to the delivery of the selected performance improvement strategy through the output user interface of the available IoT device during the speaking performance;

add the detected user response to the selected performance improvement strategy to the speaking performance profile of the user; and

communicate, responsive to the detected response of the user to the delivery of the selected performance improvement strategy being a negative response, instructions to deliver a next best performance improvement strategy according to the ranking to one of the available IoT devices during the speaking performance.

19. The computer program product of claim 17, the computer readable storage device further comprising instructions to:

generate a public speaking performance scenario having a set of parameter constraints;

communicate the public speaking performance scenario to the user; and

monitor the public speaking performance of the user under the set of parameter constraints.

20. The computer program product of claim 17, the computer readable storage device further comprising instructions to use a machine learning algorithm to generate the speaking performance profile of the user based on previous speaking performance interactions of the user with IoT devices delivering performance improvement strategies.