Systems and Methods for Providing a Multi-Modal Evaluation of a Presentation

• US 20140302469A1
• Filed: 04/08/2014
• Published: 10/09/2014
• Est. Priority Date: 04/08/2013
• Status: Active Grant

• Current Assignee: Educational Testing Service
• Original Assignee: Educational Testing Service
• Inventors: Chen, Lei, Feng, Gary, Leong, Chee Wee, Kitchen, Christopher, Lee, Chong Min
• Granted Patent: US 10,311,743 B2
• Time in Patent Office: Days
• US Class Current: 434/236
• CPC Class Codes: G09B 19/00 Teaching not covered by other main groups of this subclass teaching or practice apparatus for gun-aiming or gun-laying F41G3/26

Abstract

Systems and methods are described for providing a multi-modal evaluation of a presentation. A system includes a motion capture device configured to detect motion an examinee giving a presentation and an audio recording device configured to capture audio of the examinee giving the presentation. One or more data processors are configured to extract a non-verbal feature of the presentation based on data collected by the motion capture device and an audio feature of the presentation based on data collected by the audio recording device. The one or more data processors are further configured to generate a presentation score based on the non-verbal feature and the audio feature.

Claims

1. A computer-implemented system for providing a multi-modal evaluation of a presentation, comprising:

a motion capture device configured to detect motion an examinee giving a presentation;

an audio recording device configured to capture audio of the examinee giving the presentation; and

a processing system configured to;

extract a non-verbal feature of the presentation based on data collected by the motion capture device;

extract an audio feature of the presentation based on data collected by the audio recording device; and

generate a presentation score based on the non-verbal feature and the audio feature.

2. The system of claim 1, wherein the motion capture device is a three dimensional depth measurement device or a marker based motion detection device.

3. The system of claim 2, wherein the non-verbal feature includes a measurement of gestures made by the examinee during the presentation or a measurement of a posture of the examinee during the presentation.

4. The system of claim 1, wherein the motion capture device includes a video camera;

wherein the non-verbal feature includes a measurement of eye contact of the examinee during the presentation;

wherein the non-verbal feature further includes a measurement of facial expression by the examinee during the presentation.

5. The system of claim 1, wherein extracting the audio feature includes:

providing the data collected by the audio recording device to an automatic speech recognizer as part of extraction of the audio feature;

wherein the automatic speech recognizer generates a transcript of speech of the presentation, wherein the one or more data processors are configured to generate the audio feature based on the transcript.

6. The system of claim 5, wherein the presentation is given based on a presentation topic prompt, wherein the audio feature is based on content in the presentation relative to the presentation topic prompt based on the transcript.

7. The system of claim 5, wherein the automatic speech recognizer generates non-content, delivery based features of the presentation based on the data collected by the audio recording device;

wherein the audio feature is a fluency or prosody measurement determined based on the non-content based features generated by the automatic speech recognizer.

8. A computer-implemented method of providing a multi-modal evaluation of a presentation, comprising:

detecting motion of an examinee giving a presentation using a motion capture device;

capturing audio of the examinee giving the presentation using an audio recording device;

extracting a non-verbal feature of the presentation based on data collected by the motion capture device;

extracting an audio feature of the presentation based on data collected by the audio recording device; and

generating a presentation score based on the non-verbal feature and the audio feature.

9. The method of claim 8, wherein the motion capture device is a three dimensional depth measurement device or a marker based motion detection device.

10. The method of claim 9, wherein the non-verbal feature includes a measurement of gestures made by the examinee during the presentation or a measurement of a posture of the examinee during the presentation.

11. The method of claim 8, wherein the motion capture device includes a video camera;

wherein the non-verbal feature includes a measurement of eye contact of the examinee during the presentation;

wherein the non-verbal feature further includes a measurement of facial expression by the examinee during the presentation.

12. The method of claim 8, wherein extracting the audio feature includes:

providing the data collected by the audio recording device to an automatic speech recognizer as part of extraction of the audio feature;

wherein the automatic speech recognizer generates a transcript of speech of the presentation, wherein the one or more data processors are configured to generate the audio feature based on the transcript.

13. The method of claim 12, wherein the presentation is given based on a presentation topic prompt, wherein the audio feature is based on content in the presentation relative to the presentation topic prompt based on the transcript.

14. The method of claim 12, wherein the automatic speech recognizer generates non-content, delivery based features of the presentation based on the data collected by the audio recording device;

wherein the audio feature is a fluency or prosody measurement determined based on the non-content based features generated by the automatic speech recognizer.

15. A computer-readable medium encoded with instructions for commanding one or more data processors to execute steps for providing a multi-modal evaluation of a presentation, comprising:

detecting audio of the examinee giving the presentation using an audio recording device;

extracting a non-verbal feature of the presentation based on data collected by the motion capture device;

extracting an audio feature of the presentation based on data collected by the audio recording device; and

generating a presentation score based on the non-verbal feature and the audio feature.

16. The computer-readable medium of claim 15, wherein the motion capture device is a three dimensional depth measurement device or a marker based motion detection device.

17. The computer-readable medium of claim 16, wherein the non-verbal feature includes a measurement of gestures made by the examinee during the presentation or a measurement of a posture of the examinee during the presentation.

18. The computer-readable medium of claim 15, wherein the motion capture device includes a video camera;

wherein the non-verbal feature includes a measurement of eye contact of the examinee during the presentation;

wherein the non-verbal feature further includes a measurement of facial expression by the examinee during the presentation.

19. The computer-readable medium of claim 15, wherein extracting the audio feature includes:

providing the data collected by the audio recording device to an automatic speech recognizer as part of extraction of the audio feature;

wherein the automatic speech recognizer generates a transcript of speech of the presentation, wherein the one or more data processors are configured to generate the audio feature based on the transcript.

20. The computer-readable medium of claim 19, wherein the presentation is given based on a presentation topic prompt, wherein the audio feature is based on content in the presentation relative to the presentation topic prompt based on the transcript.

21. The computer-readable medium of claim 20, wherein the automatic speech recognizer generates non-content, delivery based features of the presentation based on the data collected by the audio recording device;

wherein the audio feature is a fluency or prosody measurement determined based on the non-content based features generated by the automatic speech recognizer.