EXPRESSION CURVE GENERATING METHOD BASED ON VOICE INPUT AND DEVICE THEREOF

• US 20180150202A1
• Filed: 05/30/2016
• Published: 05/31/2018
• Est. Priority Date: 05/28/2015
• Status: Active Grant

• Current Assignee: Cienet Technologies Company Limited
• Original Assignee: Cienet Technologies Company Limited
• Inventors: ZHENG, Liang, LIU, Xibin
• Granted Patent: US 10,521,071 B2
• Time in Patent Office: Days
• CPC Class Codes: G06F 3/04817 using icons graphical or visual programming using iconic symbols G06F8/34
• CPC Class Codes: G06F 3/0482 Interaction with lists of selectable items, e.g. menus
• CPC Class Codes: G06F 3/04847 Interaction techniques to control parameter settings, e.g. interaction with sliders or dials
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: H04L 51/04 Real-time or near real-time messaging, e.g. instant messaging [IM]
• CPC Class Codes: H04L 51/10 Multimedia information

Abstract

Disclosed are an expression curve generating method based on voice input, and an device using the same. The method comprises the following steps: (1) inputting voice, and generating an audio file; (2) selecting expression graph buttons corresponding to a type of expressions from multiple expression graph buttons; (3) dividing the audio file into multiple equal time sections according to a time length, and respectively calculating a corresponding volume of each time section; (4) quantifying the corresponding volumes of the different time sections in the audio file into different volume levels; (5) obtaining expression icons corresponding to the volume level of each time section from a same group of expression icons, generating an expression curve using time as a horizontal axis and the volume level as a vertical axis; and (6) displaying the expression curve formed in step (5).

Claims

1. An expression curve generating method based on voice input, wherein the method comprises the following steps:

(1) performing voice input to generate an audio file;

(2) selecting an expression graph button corresponding to one type of expression from multiple expression graph buttons, wherein a single expression graph button corresponds to one or more groups of expression icons, and each group of expression icons comprises multiple expression icons respectively corresponding to different sound volume levels;

(3) dividing the audio file into multiple equal time periods according to duration, and calculating a corresponding sound volume of each time period;

(4) quantifying the corresponding sound volumes of the different time periods in the audio file into different sound volume levels;

(5) obtaining an expression icon corresponding to a sound volume level of each time period from a same group of expression icons corresponding to the expression graph button selected in step (2), and generating an expression curve using time as a horizontal axis and the sound volume level as a vertical axis, wherein the expression icon corresponding to the sound volume level is displayed in the position of the sound volume level corresponding to each time period; and

(6) displaying the expression curve formed in step (5).

2. The expression curve generating method according to claim 1, wherein the method further comprises step (7):

touching the expression graph button selected in step (2) again, changing a group of expression icons, and then returning to step (5) to reconstruct an expression curve by using the group of expression icons.

3. The expression curve generating method according to claim 2, wherein:

in step (7), when the expression graph button is touched at a single time, the multiple groups of expression icons corresponding to the expression graph button are sequentially changed.

4. The expression curve generating method according to claim 2, wherein:

in step (7), when the expression graph button is touched at a single time, the multiple groups of expression icons corresponding to the expression graph button are changed according to an input habit of a user.

5. The expression curve generating method according to claim 1, wherein in step (3), a process of dividing the audio file into N equal time periods, wherein each time period comprises m frames of audio data, and calculating sound volumes corresponding to the N time periods comprises the following steps:

(31) determining whether the maximum PCM value corresponding to the N time periods is stored, and if yes, entering step (4), or if not, performing step (32);

(32) reading a next frame of audio data, and converting the current frame of audio data into PCM codes;

(33) comparing absolute values of this set of PCM codes, to find the maximum value thereof, and storing the maximum value;

(34) determining whether all m PCM values corresponding to the m frames of audio data in the current time period have been obtained and stored, if yes, entering step (35);

or if not, returning to step (32); and

(35) comparing the m PCM values, to find the maximum value thereof, using this value as a sound volume in the time period, and then returning to step (31).

6. The expression curve generating method according to claim 1, wherein in step (4), quantifying m PCM values into four sound volume levels:

a very high decibel, a high decibel, an intermediate decibel, and a low decibel comprises the following steps;

(41) selecting the minimum PCM value as a lower limit (n1) of the low decibel, and selecting the maximum PCM value as an upper limit (n5) of the very high decibel;

(42) calculating a decibel difference between two adjacent levels;

d=(the maximum PCM value−

the minimum PCM value)/4, and calculating boundary values of decibels corresponding to the four levels;

n2=the minimum PCM value+d;

n3=the minimum PCM value+2d, and n4=the minimum PCM value+3d; and

(43) forming five parallel lines from bottom to top in an order of n1, n2, n3, n4, and n5, and separately determining a decibel interval within which a PCM value corresponding to each time period correspondingly falls, to determine sound volume levels corresponding to the m PCM values.

7. The expression curve generating method according to claim 1, wherein:

step (6) may further comprises step (60);

obtaining a monophonic tone corresponding to a sound volume level of each expression for the expression, and sequentially resonating monophonic tones corresponding to multiple expression icons one by one when the expression curve is being displayed.

8. The expression curve generating method according to claim 7, wherein:

in step (60), each group of expression icons comprises expression icons corresponding to four sound volume levels, and the expression icons are respectively paired with four tones in the Chinese pinyin.

9. The expression curve generating method according to claim 1, wherein:

in step (1), a user records, by touching a record button icon, a short voice recording to generate an audio file, and a storage type of the audio file is an AMR format file.

10. The expression curve generating method according to claim 1, wherein:

in step (2), the multiple expression graph buttons provided for a user to select are expression graph buttons corresponding to seven emotions “

pleasure”

, “

anger”

, “

sorrow”

, “

love”

, “

fear”

, “

disgust”

, and “

hatred”

.

11. An instant messaging device, comprising:

a processor; and

a memory, configured to store an executable instruction of the processor, wherein

the processor is configured to perform the expression curve generating method according to claim 1.

12. An instant messaging system, comprising a server and at least two terminals, wherein the terminal is connected to the server by using a wired network or a wireless network, wherein:

the terminal is the instant messaging device according to claim 11.

13. (canceled)