Devices and methods providing automated assistance for verbal communication

• US 7,818,179 B2
• Filed: 11/12/2004
• Issued: 10/19/2010
• Est. Priority Date: 11/12/2004
• Status: Expired due to Fees

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Krasikov, Sophia, Velayudham, Senthilnathan, Morar, John F.
• Primary Examiner(s): Hudspeth; David R
• Assistant Examiner(s): Neway; Samuel G
• Application Number:
  US10/988,013
• Publication Number:
  US 20060106611A1
• Time in Patent Office: 2,167 Days
• Field of Search: None
• US Class Current: 704/274
• CPC Class Codes: G09B 19/04 Speaking with audible presentation of the material to be studied G09B5/04
• CPC Class Codes: G10L 17/26 Recognition of special voice characteristics, e.g. for use in lie detectors; Recognition of animal voices

Abstract

Devices and methods are provided for automatically analyzing a user'"'"'s conversation or verbal presentation and providing feedback to the user, in real-time and/or off-line, to make the user aware of improper speech habits and thus enable the user to acquire better language habits while speaking. For example, a device for providing user awareness of speech habits may be a portable device, which can be worn or carried by the user, has various functions for monitoring and analyzing a speech session (e.g. conversation, verbal presentation) of a user and providing notification to the user of improper speech or language habits that are identified during the speech session.

Claims

1. A non-transitory storage device embodying instructions executable by a processor to perform an automated method for providing awareness of speech habits of a speaker using the program storage device, comprising:

receiving speech input from the speaker during a speaking session;

receiving speech input from a different speaker during the speaking session;

segmenting in real time the speech input from the speaker from the speech input from the different speaker;

processing segmented speech input from the speaker to determine that a speech habit of the speaker occurs; and

generating an alert to the speaker in real time while the speaker is speaking during the speaking session from which the speech input of the speaker and the speech input from the different speaker is segmented,

wherein the speech input from the speaker is analyzed to detect one or more words or expressions or sounds, if any, which are specified in a vocabulary list, in the speech input of the user,

wherein an identified speech habit comprises exceeding a range of volume of speaking a word or expression specified in the vocabulary list, and

wherein a counter is incremented corresponding to a number of instances of exceeding a range of volume in the speech input from the speaker and the counter is compared to a repetition threshold for determining a speech habit in the speech input from the speaker based upon a predetermined value of the counter within a predetermined time period.

2. The non-transitory storage device of claim 1, further comprising:

processing the speech input from the speaker to determine an instance of a predetermined filler in the speech input, wherein the predetermined filler is specified in a vocabulary list;

incrementing a counter corresponding to a number of instances of the predetermined filler in the speech input from the speaker;

comparing the counter to a repetition threshold for determining a speech habit in the speech input from the speaker, wherein the speech habit is determined upon a predetermined value of the counter within a predetermined time period; and

collecting data associated with alerts generated during the speaking session; and

generating a session report based on the data after a termination of the speaking session.

3. The non-transitory storage device of claim 1, wherein:

speech processing results obtained during the speaking session are storable, and

analysis of the speech processing results is performable after the speaking session has ended,

wherein a predetermined number of spoken words before and after a detected word/expression or sound in the storable speech processing results is recorded to provide a context for the detected word/expression or sound.

4. A device to provide awareness of speech habits of a speaker using the device, comprising:

an audio input device;

a speech processing system that processes speech input from the speaker through the audio input device, segments in real time speech input from a different speaker through the audio input device, and provides speech processing results;

a language analysis system that analyzes the speech processing results output from the speech processing system using pre-specified criteria for identifying a speech habit of the speaker;

an alert system that alerts the speaker in real time while the speaker is speaking during the speaking session from which the speech input of the speaker and the speech input of the different speaker is segmented; and

a user interface for controlling the device,

wherein the speech processing system comprises a word spotting system adapted to analyze the speech input of a user for detecting one or more words or expressions or sounds, if an which are specified in a vocabulary list, in the speech input of the user, and

wherein an identified speech habit comprises exceeding a range of volume of speaking a word or expression specified in the vocabulary list, and

wherein a counter is incremented corresponding to a number of instances of exceeding a range of volume in the speech input from the speaker and the counter is compared to a repetition threshold for determining a speech habit in the speech input from the speaker based upon a predetermined value of the counter within a predetermined time period.

5. The device of claim 4, wherein the device is wearable by a user.

6. The device of claim 4, wherein the speech processing system comprises a word spotting system adapted to analyze the speech input of a user for detecting one or more words or expressions or sounds, if any, which are specified in a vocabulary list, in the speech input of the user.

7. The device of claim 6, wherein an identified speech habit comprises an incorrect pronunciation of a word.

8. The device of claim 6, wherein an identified speech habit comprises a repetitive use of a word or expression specified in the vocabulary list.

9. The device of claim 6, wherein an identified speech habit comprises a sound specified in the vocabulary list being made by the user.

10. The device of claim 6, wherein the speech processing system comprises a learning methodology system that automatically compiles the vocabulary list from speech data collected during usage of the device.

11. The device of claim 6, wherein the vocabulary list is editable by the user through the user interface.

12. The device of claim 4, wherein a tactile alert signal is provided by the alert system.

13. The device of claim 12, wherein the tactile alert signal comprises a vibration.

14. The device of claim 12, wherein the tactile alert signal comprises a pin prick.

15. The device of claim 4, wherein the alert system provides an audio alert signal.

16. The device of claim 4, wherein the user interface is adapted to allow the user to specify an alert type provided by the alert system.

17. The device of claim 4, wherein:

the device is operable in an “

off-line”

mode,

the speech processing results obtained during a user speaking session are storable, and

analysis of the speech processing results is performable after the user speaking session has ended.

18. The device of claim 4, further comprising at least one sensor that collects environmental data during a user speaking session, wherein the language analysis system is adapted to use the collected environmental data for identifying a speech habit or for evaluating an identified speech habit.

19. The device of claim 18, wherein the at least one sensor includes a temperature sensor, a lighting sensor, or a time sensor.

20. The device of claim 4, wherein:

the speech processing results obtained during the speaking session are storable, and

analysis of the speech processing results is performable after the speaking session has ended,

wherein a predetermined number of spoken words before and after a detected word/expression or sound in the storable speech processing results is recorded to provide a context for the detected word/expression or sound.

21. An automated method for providing awareness of speech habits of a speaker comprising:

providing a speech processing system wearable or hand-held by the speaker;

processing by the speech processing system speech input from the speaker during a speaking session;

processing by the speech processing system speech input from a different speaker during the speaking session;

segmenting in real time by the speech processing system the speech input from the speaker from the speech input from the different speaker;

analyzing by the speech processing system speech processing results of the speaker using pre-specified criteria to identify a speech habit of the speaker; and

alerting the speaker in real time while the speaker is speaking during the speaking session from which the speech input of the speaker and the speech input of the different speaker is segmented,

wherein the speech input of the speaker is analyzed to detect one or more words or expressions or sounds, if any, which are specified in a vocabulary list, in the speech input of the user,

wherein an identified speech habit comprises exceeding a range of volume of speaking a word or expression specified in the vocabulary list, and

wherein a counter is incremented corresponding to a number of instances of exceeding a range of volume in the speech input from the speaker and the counter is compared to a repetition threshold for determining a speech habit in the speech input from the speaker based upon a predetermined value of the counter within a predetermined time period.

22. The method of claim 21, wherein processing speech input from the speaker comprises analyzing the speech input of the speaker for detecting one or more words or expressions or sounds, if any, in the speech input, which are specified in a vocabulary list.

23. The method of claim 22, wherein an identified speech habit comprises an incorrect pronunciation of a word.

24. The method of claim 22, wherein an identified speech habit comprises a repetitive use of a word or expression specified in the vocabulary list.

25. The method of claim 22, wherein an identified speech habit comprises sounds specified in the vocabulary list being made by the speaker.

26. The method of claim 22, further comprising automatically compiling the vocabulary list from speech data collected from the speaker.

27. The method of claim 21, wherein alerting the speaker comprises providing a tactile alert signal.

28. The method of claim 27, wherein the tactile alert signal comprises a vibration.

29. The method of claim 27, wherein the tactile alert signal comprises a pin prick.

30. The method of claim 21, wherein alerting the speaker comprises providing an audio alert signal.

31. The method of claim 21, further comprising manually specifying an alert signal by the speaker.

32. The method of claim 21, further comprising storing speech processing results obtained during a user speaking session, and wherein analyzing the speech processing results is performed after the speaking session has ended.

33. The method of claim 21, further comprising collecting environmental data during the speaking session, wherein analyzing further comprises using collected environmental data for identifying a speech habit or for evaluating an identified speech habit.

34. The method of claim 33, wherein the collected environmental data comprises one of temperature, lighting, time, or a combination thereof.

35. The automated method of claim 21, wherein:

speech processing results obtained during the speaking session are storable, and

analysis of the speech processing results is performable after the speaking session has ended,

wherein a predetermined number of spoken words before and after a detected word/expression or sound in the storable speech processing results is recorded to provide a context for the detected word/expression or sound.