Voice synthesizing having vocalization according to user manipulation

• US 9,424,831 B2
• Filed: 02/20/2014
• Issued: 08/23/2016
• Est. Priority Date: 02/22/2013
• Status: Active Grant

• Current Assignee: Yamaha Corporation
• Original Assignee: Yamaha Corporation
• Inventors: Hisaminato, Yuji
• Primary Examiner(s): SINGH, SATWANT K
• Application Number:
  US14/185,448
• Publication Number:
  US 20140244262A1
• Time in Patent Office: 915 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: G10H 1/0091 Means for obtaining special acoustic effects combined with modulation G10H1/043
• CPC Class Codes: G10H 1/14 during execution voice controlled instruments G10H5/005
• CPC Class Codes: G10H 2220/096 using a touch screen touch screen note input, e.g. using a displayed keyboard G10H2220/241; personal digital assistant [PDA] G10H2230/015
• CPC Class Codes: G10H 2250/455 Gensound singing voices, i.e. generation of human voices for musical applications, vocal singing sounds or intelligible words at a desired pitch or with desired vocal effects, e.g. by phoneme synthesis formant synthesis G10H2250/481; parcor synthesis G10H2250/505; modulation effects G10H2210/195; ensemble effects G10H2210/245; speech synthesis in general G10L13/00
• CPC Class Codes: G10H 7/008 Means for controlling the transition from one tone waveform to another
• CPC Class Codes: G10L 13/02 Methods for producing synthetic speech; Speech synthesisers
• CPC Class Codes: G10L 13/07 Concatenation rules

Abstract

A voice synthesizing apparatus includes a manipulation determiner configured to determine a manipulation position which is moved according to a manipulation of a user, and a voice synthesizer configured to generate, in response to an instruction to generate a voice in which a second phoneme follows a first phoneme, a voice signal so that vocalization of the first phoneme starts before the manipulation position reaches a reference position and that vocalization from the first phoneme to the second phoneme is made when the manipulation position reaches the reference position.

Claims

1. A voice synthesizing method comprising:

determining a manipulation position which is moved according to a user'"'"'s manipulation for moving the manipulation position, wherein the user'"'"'s manipulation is made on a manipulation device;

generating, in response to an instruction to generate a voice in which a second phoneme follows a first phoneme, a voice signal so that vocalization of the first phoneme starts before the moving manipulation position reaches a reference position and that vocalization from the first phoneme to the second phoneme is made when the moving manipulation position reaches the reference position; and

predicting an instruction time point when the moving manipulation position reaches the reference position on the basis of a movement speed of the moving manipulation position.

2. The voice synthesizing method according to claim 1, wherein, in the predicting, the instruction time point is predicted on the basis of a time length that the moving manipulation position takes to move from a prediction start position to a prediction execution position.

3. The voice synthesizing method according to claim 2, wherein the prediction execution position is variably set in accordance with a kind of the first phoneme.

4. The voice synthesizing method according to claim 1, wherein, in the generating, the generated voice signal is a voice signal for vocalizing a synthesis unit at a time point when the moving manipulation position that is moving toward the reference position passes a vocalization start position, the synthesis unit having the first phoneme.

5. The voice synthesizing method according to claim 4, wherein the vocalization start position is variably set in accordance with a kind of the first phoneme.

6. The voice synthesizing method according to claim 1, wherein, in the generating, the generated voice signal is a voice signal having an acoustic effect based on movement of the moving manipulation position in a direction that crosses the manipulation path, the manipulation path extending in another direction toward the reference position.

7. The voice synthesizing method according to claim 1,

wherein, in the generating, the instruction to generate the voice is an instruction to generate a voice in which the second phoneme follows the first phoneme and a voice in which a fourth phoneme follows a third phoneme,

wherein, the generating is generating, in response to the instruction:

the voice signal so that the vocalization of the first phoneme starts before the moving manipulation position reaches the reference position as a first reference position as a result of movement of the moving manipulation position along a manipulation path in a first direction and that the vocalization from the first phoneme to the second phoneme is made when the moving manipulation position reaches the first reference position; and

a voice signal so that vocalization of the third phoneme starts before the moving manipulation position reaches a second reference position as a result of movement of the moving manipulation position along the manipulation path in a second direction that is opposite to the first direction and that vocalization from the third phoneme to the fourth phoneme is made when the moving manipulation position reaches the second reference position.

8. A voice synthesizing method comprising:

determining a manipulation position which is moved according to a user'"'"'s manipulation for moving the manipulation position, wherein the user'"'"'s manipulation is made on a manipulation device;

generating, in response to an instruction to generate a voice in which a second phoneme follows a first phoneme, a voice signal so that vocalization of the first phoneme starts before the moving manipulation position reaches a reference position and that vocalization from the first phoneme to the second phoneme is made when the moving manipulation position reaches the reference position,

wherein, in the generating, the generated voice signal is a voice signal having a pitch that corresponds to a manipulation path along which the moving manipulation position is moved, among plural manipulation paths corresponding to different pitches.

9. A voice synthesizing method comprising:

determining a manipulation position which is moved according to a user'"'"'s manipulation for moving the manipulation position, wherein the user'"'"'s manipulation is made on a manipulation device;

generating, in response to an instruction to generate a voice in which a second phoneme follows a first phoneme, a voice signal so that vocalization of the first phoneme starts before the moving manipulation position reaches a reference position and that vocalization from the first phoneme to the second phoneme is made when the moving manipulation position reaches the reference position,

wherein, in the generating, the generated voice signal is a voice signal for a vocalization code that corresponds to a manipulation path, along which the moving manipulation position is moved, among plural manipulation paths corresponding to different vocalization codes.

10. A voice synthesizing method comprising:

determining a manipulation position which is moved according to a user'"'"'s manipulation for moving the manipulation position, wherein the user'"'"'s manipulation is made on a manipulation device;

generating, in response to an instruction to generate a voice in which a second phoneme follows a first phoneme, a voice signal so that vocalization of the first phoneme starts before the moving manipulation position reaches a reference position and that vocalization from the first phoneme to the second phoneme is made when the moving manipulation position reaches the reference position,

wherein, in the generating, the generated voice signal is a voice signal having a pitch that corresponds to where the moving manipulation position is located in a direction that crosses a manipulation path, the manipulation path extending in another direction toward the reference position.

11. A voice synthesizing apparatus comprising:

a non-transitory recording medium configured to store one or more programs;

a computer, when executing the one or more programs, configured to;

determine a manipulation position which is moved according to a user'"'"'s manipulation for moving the manipulation position, wherein the user'"'"'s manipulation is made on a manipulation device;

generate, in response to an instruction to generate a voice in which a second phoneme follows a first phoneme, a voice signal so that vocalization of the first phoneme starts before the moving manipulation position reaches a reference position and that vocalization from the first phoneme to the second phoneme is made when the moving manipulation position reaches the reference position; and

predict an instruction time point when the moving manipulation position reaches the reference position on the basis of a movement speed of the moving manipulation position.

12. The voice synthesizing apparatus according to claim 11, wherein the instruction time point is predicted on the basis of a time length that the moving manipulation position takes to move from a prediction start position to a prediction execution position.

13. The voice synthesizing apparatus according to claim 12, wherein the computer, when executing the one or more programs, manipulation predictor is configured to set the prediction execution position variably in accordance with a kind of the first phoneme.

14. The voice synthesizing apparatus according to claim 11, wherein the generated voice signal is a voice signal for vocalizing a synthesis unit at a time point when the moving manipulation position that is moving toward the reference position passes a vocalization start position, the synthesis unit having the first phoneme.

15. The voice synthesizing apparatus according to claim 14, wherein the computer, when executing the one or more programs, is configured to set the vocalization start position variably in accordance with a kind of the first phoneme.

16. The voice synthesizing apparatus according to claim 11, wherein the generated voice signal is a voice signal having an acoustic effect based on movement of the moving manipulation position in a direction that crosses the manipulation path, the manipulation path extending in another direction toward the reference position.

17. The voice synthesizing apparatus according to claim 11,

wherein the instruction to generate the voice is an instruction to generate a voice in which the second phoneme follows the first phoneme and a voice in which a fourth phoneme follows a third phoneme,

wherein the computer, when executing the one or more programs, is configured to generate:

the voice signal so that the vocalization of the first phoneme starts before the moving manipulation position reaches the reference position as a first reference position as a result of movement of the moving manipulation position along a manipulation path in a first direction and that the vocalization from the first phoneme to the second phoneme is made when the moving manipulation position reaches the first reference position; and

a voice signal so that vocalization of the third phoneme starts before the moving manipulation position reaches a second reference position as a result of movement of the moving manipulation position along the manipulation path in a second direction that is opposite to the first direction and that vocalization from the third phoneme to the fourth phoneme is made when the moving manipulation position reaches the second reference position.

18. A voice synthesizing apparatus comprising:

a non-transitory recording medium configured to store one or more programs;

a computer, when executing the one or more programs, configured to;

determine a manipulation position which is moved according to a user'"'"'s manipulation for moving the manipulation position, wherein the user'"'"'s manipulation is made on a manipulation device; and

generate, in response to an instruction to generate a voice in which a second phoneme follows a first phoneme, a voice signal so that vocalization of the first phoneme starts before the moving manipulation position reaches a reference position and that vocalization from the first phoneme to the second phoneme is made when the moving manipulation position reaches the reference position,

wherein the generated voice signal is a voice signal having a pitch that corresponds to a manipulation path along which the moving manipulation position is moved, among plural manipulation paths corresponding to different pitches.

19. A voice synthesizing apparatus comprising:

a non-transitory recording medium configured to store one or more programs;

a computer, when executing the one or more programs, configured to;

determine a manipulation position which is moved according to a user'"'"'s manipulation for moving the manipulation position, wherein the user'"'"'s manipulation is made on a manipulation device; and

generate, in response to an instruction to generate a voice in which a second phoneme follows a first phoneme, a voice signal so that vocalization of the first phoneme starts before the moving manipulation position reaches a reference position and that vocalization from the first phoneme to the second phoneme is made when the moving manipulation position reaches the reference position,

wherein the generated voice signal is a voice signal for a vocalization code that corresponds to a manipulation path along which the moving manipulation position is moved, among plural manipulation paths corresponding to different vocalization codes.

20. A voice synthesizing apparatus comprising:

a non-transitory recording medium configured to store one or more programs;

a computer, when executing the one or more programs, configured to;

determine a manipulation position which is moved according to a user'"'"'s manipulation for moving the manipulation position, wherein the user'"'"'s manipulation is made on a manipulation device; and

generate, in response to an instruction to generate a voice in which a second phoneme follows a first phoneme, a voice signal so that vocalization of the first phoneme starts before the moving manipulation position reaches a reference position and that vocalization from the first phoneme to the second phoneme is made when the moving manipulation position reaches the reference position,

wherein the generated voice signal is a voice signal having a pitch that corresponds to where the moving manipulation position is located in a direction that crosses a manipulation path, the manipulation path extending in another direction toward the reference position.