Language delay treatment system and control method for the same

• US 9,875,668 B2
• Filed: 10/07/2013
• Issued: 01/23/2018
• Est. Priority Date: 09/05/2013
• Status: Active Grant

• Current Assignee: Korea Advanced Institute of Science and Technology
• Original Assignee: Korea Advanced Institute of Science and Technology
• Inventors: Song, June Hwa, Hwang, In Seok, Yoo, Chung Kuk, Hwang, Chan You, Lee, Young Ki, Kim, John Dong Jun, Yim, Dong Sun Jennifer, Min, Chul Hong
• Primary Examiner(s): Yao, Sam
• Assistant Examiner(s): Carlos, Alvin
• Application Number:
  US14/047,177
• Publication Number:
  US 20150064666A1
• Time in Patent Office: 1,569 Days
• Field of Search: 434185
• CPC Class Codes: G09B 19/04 Speaking with audible presentation of the material to be studied G09B5/04
• CPC Class Codes: G09B 5/04 with audible presentation of the material to be studied sound recording or reproducing G11B
• CPC Class Codes: G09B 5/14 with provision for individual teacher-student communication
• CPC Class Codes: G10L 25/78 Detection of presence or absence of voice signals switching of direction of transmission by voice frequency in two-way loud-speaking telephone systems H04M9/10

Abstract

The present disclosures relates to a control terminal, comprising: a data communication unit for receiving a first user voice by data communication with a first audio device and receiving a second user voice by data communication with a second audio device; a turn information generating unit for generating turn information, which is voice unit information, by using the first and second user voices; and a metalanguage processing unit for determining a conversation pattern of the first and second users by using the turn information, and outputting a reminder message corresponding to a reminder event to the first user when the conversation pattern corresponds to a preset reminder event occurrence condition.

Claims

1. A control terminal comprising:

a data communication unit for receiving a first user voice by data communication with a first audio device and receiving a second user voice by data communication with a second audio device;

a turn information generating unit for generating turn information, which is voice unit information, by using the first and second user voices; and

a meta-language processing unit for determining a conversation pattern of the first and second users by using the turn information, and outputting a reminder message corresponding to a reminder event to the first user when the conversation pattern corresponds to a preset reminder event occurrence condition,

wherein the reminder event occurrence condition includes;

a first reminder event occurrence condition, which is triggered when the first user'"'"'s turn repeats N_dominance times in which pauses between adjacent turns are shorter than T_wait and no second user'"'"'s turns appear during the first user'"'"'s turns, wherein

N_dominance represents a preset repetition number of the first user'"'"'s turns, and

T_wait represents a preset interval time of the first user'"'"'s turns;

a second reminder event occurrence condition, which is triggered when for the second user'"'"'s given turn, neither the first nor the second user'"'"'s turns appear within T_neglect, for N_grace2 times, wherein

N_grace2 represents a preset repetition number of the second user'"'"'s turns, and

T_neglect represents a preset interval time of the second user'"'"'s turns;

a third reminder event occurrence condition, which is triggered when the first user'"'"'s turns begin before the second user'"'"'s turns end, for N_grace3 times, wherein

N_grace3 represents a preset number of the first user'"'"'s turns which occur before the second user'"'"'s turns end;

a fourth reminder event occurrence condition, which is triggered when a duration time of the first user'"'"'s turns is longer than T_long and no second user'"'"'s turns appear within T_response4 after the first user'"'"'s turns ends, wherein

T_long represents a preset duration time of the first user'"'"'s turns, and

T_response4 represents a preset generation time of the second user'"'"'s turns; and

a fifth reminder event occurrence condition, which is triggered when a syllable rate of the first user'"'"'s turns is higher than R_fast and no second user'"'"'s turns appear within T_response5 after the first user'"'"'s turns end, wherein

R_fast represents a preset voice speed, and

T_response5 represents a preset generation time of the second user'"'"'s turns.

2. The control terminal of claim 1, further comprising a preprocessing unit for optionally processing the first and second user voices with respect to a voice range.

3. The control terminal of claim 1, wherein the turn information includes at least one of speaker identification information, time, accent, loudness and speed of a unit voice.

4. The control terminal of claim 3, wherein the turn information generating unit determines speaker identification information of the turn information according to a ratio of the first user voice and the second user voice.

5. The control terminal of claim 1, wherein the turn information generating unit generates the turn information when the first user voice or the second user voice is equal to or greater than a preset loudness.

6. The control terminal of claim 1, wherein the reminder event occurrence condition includes at least one of a case in which only a turn of the first user occurs during a preset time, a case in which only a turn of the second user occurs during a preset time, a case in which the turn of the first user occurs over a preset number before the turn of the second user ends, a case in which the turn of the first user continues over a preset time, and a case in which the turn of the first user is equal to or greater than a preset speed.

7. The control terminal of claim 2, wherein the turn information includes at least one of speaker identification information, time, accent, loudness and speed of a unit voice.

8. The control terminal of claim 7, wherein the turn information generating unit determines speaker identification information of the turn information according to a ratio of the first user voice and the second user voice.

9. The control terminal of claim 2, wherein the turn information generating unit generates the turn information when the first user voice or the second user voice is equal to or greater than a preset loudness.

10. The control terminal of claim 2, wherein the reminder event occurrence condition includes at least one of a case in which only a turn of the first user occurs during a preset time, a case in which only a turn of the second user occurs during a preset time, a case in which the turn of the first user occurs over a preset number before the turn of the second user ends, a case in which the turn of the first user continues over a preset time, and a case in which the turn of the first user is equal to or greater than a preset speed.

11. A control method for a language delay treatment system, which includes a first audio device for receiving a voice of a first user, a second audio device for receiving a voice of a second user, and a control terminal, the control method comprising:

receiving, by the control terminal, the first user voice by data communication with the first audio device;

receiving, by the control terminal, the second user voice by data communication with the second audio device;

generating, by the control terminal, turn information which is voice unit information by using the first and second user voices;

determining, by the control terminal, a conversation pattern of the first and second users by using the turn information; and

outputting, by the control terminal, a reminder message corresponding to a reminder event to the first user when the conversation pattern corresponds to a preset reminder event occurrence condition, which includes;

a first reminder event occurrence condition, which is triggered when the first user'"'"'s turn repeats N_dominance times in which pauses between adjacent turns are shorter than T_wait and no second user'"'"'s turns appear during the first user'"'"'s turns, wherein

N_dominance represents a preset repetition number of the first user'"'"'s turns, and

T_wait represents a preset interval time of the first user'"'"'s turns;

a second reminder event occurrence condition, which is triggered when for the second user'"'"'s given turn, neither the first nor the second user'"'"'s turns appear within T_neglect, for N_grace2 times, wherein

N_grace2 represents a preset repetition number of the second user'"'"'s turns, and

T_neglect represents a preset interval time of the second user'"'"'s turns;

a third reminder event occurrence condition, which is triggered when the first user'"'"'s turns begin before the second user'"'"'s turns end, for N_grace3 times, wherein

N_grace3 represents a preset number of the first user'"'"'s turns which occur before the second user'"'"'s turns end;

a fourth reminder event occurrence condition, which is triggered when a duration time of the first user'"'"'s turns is longer than T_long and no second user'"'"'s turns appear within T_response4 after the first user'"'"'s turns ends, wherein

T_long represents a preset duration time of the first user'"'"'s turns, and

T_response4 represents a preset generation time of the second user'"'"'s turns; and

a fifth reminder event occurrence condition, which is triggered when a syllable rate of the first user'"'"'s turns is higher than R_fast and no second user'"'"'s turns appear within T_response5 after the first user'"'"'s turns end, wherein

R_fast represents a preset voice speed, and

T_response5 represents a preset generation time of the second user'"'"'s turns;

wherein the meta-language processing unit does not output the reminder message to the second user.

12. The control method for a language delay treatment system of claim 11, further comprising:

preprocessing for optionally processing the first and second user voices with respect to a voice range.

13. The control method for a language delay treatment system of claim 11, wherein the turn information includes at least one of speaker identification information, time, accent, loudness and speed of a unit voice.

14. The control method for a language delay treatment system of claim 13, wherein the generating of turn information determines speaker identification information of the turn information according to a ratio of the first user voice and the second user voice.

15. The control method for a language delay treatment system of claim 11, wherein the generating of turn information generates the turn information when the first user voice or the second user voice is equal to or greater than a preset loudness.

16. The control method for a language delay treatment system of claim 11, wherein the reminder event occurrence condition includes at least one of a case in which only a turn of the first user occurs during a preset time, a case in which only a turn of the second user occurs during a preset time, a case in which the turn of the first user occurs over a preset number before the turn of the second user ends, a case in which the turn of the first user continues over a preset time, and a case in which the turn of the first user is equal to or greater than a preset speed.

17. The control method for a language delay treatment system of claim 12, wherein the turn information includes at least one of speaker identification information, time, accent, loudness and speed of a unit voice.

18. The control method for a language delay treatment system of claim 17, wherein the generating of turn information determines speaker identification information of the turn information according to a ratio of the first user voice and the second user voice.

19. The control method for a language delay treatment system of claim 12, wherein the generating of turn information generates the turn information when the first user voice or the second user voice is equal to or greater than a preset loudness.

20. The control method for a language delay treatment system of claim 12, wherein the reminder event occurrence condition includes at least one of a case in which only a turn of the first user occurs during a preset time, a case in which only a turn of the second user occurs during a preset time, a case in which the turn of the first user occurs over a preset number before the turn of the second user ends, a case in which the turn of the first user continues over a preset time, and a case in which the turn of the first user is equal to or greater than a preset speed.

21. A control terminal comprising:

a data communication unit for receiving a first user voice by data communication with a first audio device and receiving a second user voice by data communication with a second audio device;

a turn information generating unit for generating turn information, which is voice unit information, by using the first and second user voices; and

a meta-language processing unit for determining a conversation pattern of the first and second users by using the turn information, and outputting a reminder message corresponding to a reminder event to the first user when the conversation pattern corresponds to a preset reminder event occurrence condition, wherein the meta-language processing unit comprises a turn buffer and a reminder message trigger, which is activated upon a match between the conversation pattern stored in the buffer corresponds to the preset reminder event occurrence condition, and wherein the metalanguage processing unit does not output the reminder message to the second user;

wherein the reminder event occurrence condition includes;

a first reminder event occurrence condition, which is triggered when the first user'"'"'s turn repeats N_dominance times in which pauses between adjacent turns are shorter than T_wait and no second user'"'"'s turns appear during the first user'"'"'s turns, wherein

N_dominance represents a preset repetition number of the first user'"'"'s turns, and

T_wait represents a preset interval time of the first user'"'"'s turns;

a second reminder event occurrence condition, which is triggered when for the second user'"'"'s given turn, neither the first nor the second user'"'"'s turns appear within T_neglect, for N_grace2 times, wherein

N_grace2 represents a preset repetition number of the second user'"'"'s turns, and

T_neglect represents a preset interval time of the second user'"'"'s turns;

a third reminder event occurrence condition, which is triggered when the first user'"'"'s turns begin before the second user'"'"'s turns end, for N_grace3 times, wherein

N_grace3 represents a preset number of the first user'"'"'s turns which occur before the second user'"'"'s turns end;

a fourth reminder event occurrence condition, which is triggered when a duration time of the first user'"'"'s turns is longer than T_long and no second user'"'"'s turns appear within T_response4 after the first user'"'"'s turns ends, wherein

T_long represents a preset duration time of the first user'"'"'s turns, and

T_response4 represents a preset generation time of the second user'"'"'s turns; and

a fifth reminder event occurrence condition, which is triggered when a syllable rate of the first user'"'"'s turns is higher than R_fast and no second user'"'"'s turns appear within T_response5 after the first user'"'"'s turns end, wherein

R_fast represents a preset voice speed, and

T_response5 represents a preset generation time of the second user'"'"'s turns.