Apparatus and method of sound modulation using touch screen with pressure sensor

• US 9,997,148 B2
• Filed: 10/18/2016
• Issued: 06/12/2018
• Est. Priority Date: 10/20/2015
• Status: Active Grant

• Current Assignee: Industry Academic Cooperation Foundation
• Original Assignee: Industry Academic Cooperation Foundation
• Inventors: Cho, Kwang Su, Choi, Chang Kyu, Nam, Choon Sung, Kang, Min Gu
• Primary Examiner(s): Donels, Jeffrey
• Application Number:
  US15/297,010
• Publication Number:
  US 20170110101A1
• Time in Patent Office: 602 Days
• Field of Search: 84615
• CPC Class Codes: G06F 3/0414 using force sensing means to determine a position
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: G10H 1/182 Key multiplexing G10H1/185 takes precedence
• CPC Class Codes: G10H 1/34 Switch arrangements, e.g. keyboards or mechanical switches specially adapted for electrophonic musical instruments
• CPC Class Codes: G10H 1/44 Tuning means
• CPC Class Codes: G10H 1/46 Volume control
• CPC Class Codes: G10H 2210/201 Vibrato, i.e. rapid, repetitive and smooth variation of amplitude, pitch or timbre within a note or chord
• CPC Class Codes: G10H 2210/221 Glissando, i.e. pitch smoothly sliding from one note to another, e.g. gliss, glide, slide, bend, smear or sweep
• CPC Class Codes: G10H 2220/241 on touchscreens, i.e. keys, frets, strings, tablature or staff displayed on a touchscreen display for note input purposes
• CPC Class Codes: G10H 2220/271 Velocity sensing for individual keys, e.g. by placing sensors at different points along the kinematic path for individual key velocity estimation by delay measurement between adjacent sensor signals

Abstract

Disclosed are an apparatus and a method of generating a sound by using a touch screen. A sound modulation apparatus according to the present invention includes: a sensor information input unit configured to receive sensing information on a position, at which a touch input of a user is applied to the screen, and pressure according to the touch input applied to the screen; and a sound modulation unit configured to set a tone frequency and volume of a sound to be output, and set the tone frequency according to the position, to which the touch input of the user is applied, and set the volume according to a size of the pressure according to the touch input.

Claims

1. A sound modulation apparatus, comprising:

a sensor information input unit configured to receive sensing information on a position, at which a touch input of a user is applied to a screen, and a pressure of the touch input applied to the screen; and

a sound modulation unit configured to set a tone frequency and volume of a sound to be output, and set the tone frequency according to the position, to which the touch input of the user is applied, and set the volume according to a size of the pressure of the touch input,

wherein the sound modulation unit includes a volume controlling unit, which determines a reference pressure according to the pressure of the touch input at a predetermined reference time, and sets the volume according to a size of the determined reference pressure, and

wherein the sound modulation unit further includes a frequency controlling unit, which sets the tone frequency according to the position, to which the touch input is applied, and changes the tone frequency according to a change in a size of the pressure of the touch input.

2. The sound modulation apparatus of claim 1, further comprising:

the screen configured to display an image.

3. The sound modulation apparatus of claim 1, wherein the pressure sensor unit includes a sensing grid corresponding to a surface of the screen,

detects and outputs the information on the position, at which the touch input of the user is applied to the screen, by using the sensing grid, and

detects and outputs the information on the pressure at the position, to which the touch input of the user is applied, by using a pressure sensor provided in the sensing grid.

4. The sound modulation apparatus of claim 1, wherein the volume controlling unit determines the pressure of the touch input at the reference time as the reference pressure, in which the reference time is a time point after a predetermined time based on a time point, at which the size of the pressure of the touch input is increased to a predetermined size or more.

5. The sound modulation apparatus of claim 1, wherein the volume controlling unit determines the reference pressure by analyzing the pressure of the touch input during the predetermined reference time from a time point, at which a size of the pressure of the touch input is increased to a predetermined size or more.

6. The sound modulation apparatus of claim 4, wherein the sound modulation unit includes a setting unit which analyzes a pressure input for each time according to the touch input based on a time point, at which a size of the pressure of the touch input is increased to a predetermined size or more, and sets the reference time based on the analyzed pressure input.

7. The sound modulation apparatus of claim 1, wherein the frequency controlling unit increases or decreases the tone frequency within a predetermined frequency range based on the set tone frequency according to the change in the size of the pressure of the touch input.

8. The sound modulation apparatus of claim 7, wherein the frequency controlling unit repeatedly increases or decreases the tone frequency over time, within the predetermined frequency range based on the set tone frequency, according to the change in the size of the pressure of the touch input for vibrato of the sound to be output.

9. The sound modulation apparatus of claim 1, wherein the frequency controlling unit changes the set tone frequency according to the change in the size of the pressure after the predetermined reference time.

10. The sound modulation apparatus of claim 1, wherein when the size of the pressure is increased, the frequency controlling unit increases the tone frequency to be higher than the set tone frequency, and when the size of the pressure is decreased, the frequency controlling unit decreases the tone frequency to be lower than the set tone frequency.

11. The sound modulation apparatus of claim 1, wherein the sound modulation unit includes a multi-channel output ratio controlling unit, which recognizes a direction of the touch input by using the position, to which the touch input is applied, or the pressure of the touch input, and sets an output ratio between a plurality of channels, which are to output the sound, according to the recognized direction of the touch input.

12. The sound modulation apparatus of claim 11, wherein the multi-channel output ratio controlling unit recognizes the direction of the touch input by using information about a change in the position, to which the touch input is applied, or a change in a size of the pressure at the position.

13. The sound modulation apparatus of claim 11, wherein the multi-channel output ratio controlling unit calculates a component value for each reference direction by analyzing the recognized direction of the touch input, and sets an output ratio between the plurality of channels according to a predetermined setting value according to a size of the component value for each reference direction.

14. The sound modulation apparatus of claim 12, wherein the multi-channel output ratio controlling unit recognizes a direction of the touch input according to a direction of a movement of the position over time, to which the touch input is applied, or recognizes a direction of the touch input according to a direction, in which a distribution of sizes of the pressures at a plurality of sensing points according to the touch input is changed over time.

17. The sound modulation apparatus of claim 1, wherein when the user touches the position of the touch input after the reference time, the sound modulation unit changes the tone frequency according to a change in a size of the touch input continuously applied to the position of the touch input.

15. A sound modulation method, comprising:

a pressure sensing operation of receiving information including a detected position, at which a touch input of a user is applied to a screen, and a detected pressure of the touch input applied to the screen by using a pressure sensor;

a volume setting operation of setting, by a sound modulation unit, a volume of a sound to be output according to a size of the pressure of the touch input; and

a frequency setting operation of setting, by the sound modulation unit, a tone frequency of the sound according to the position, to which the touch input is applied,

wherein in the volume setting operation, a recognized pressure is determined according to the pressure of the touch input at a predetermined recognition time, and the volume is set according to a size of the determined recognized pressure, and

wherein in the frequency setting operation, the tone frequency is set according to the position, to which the touch input is applied, and the tone frequency is changed according to a change in a size of the pressure of the touch input.

16. The sound modulation method of claim 15, wherein the sound modulation unit further includes a multi-channel output ratio controlling operation of recognizing a direction of the touch input by using the position, to which the touch input is applied, or the pressure of the touch input, and setting an output ratio between a plurality of channels, which are to output the sound, according to the recognized direction of the touch input.

18. The sound modulation method of claim 15, wherein when the user touches the position of the touch input after the recognition time during the frequency setting operation, the tone frequency is changed according to a change in a size of the pressure of the touch input continuously applied to the position.

19. A sound modulation apparatus, comprising:

a sensor information input unit configured to receive sensing information on a position at which a touch input of a user is applied to a screen, and a pressure of the touch input applied to the screen; and

a sound modulation unit configured to set a tone frequency and volume of a sound to be output, to set the tone frequency according to the position to which the touch input of the user is applied, and to set the volume according to a size of the pressure of the touch input,

wherein the sound modulation unit includes a volume controlling unit, which determines a reference pressure according to the pressure of the touch input at a predetermined reference time, and sets the volume according to a size of the determined reference pressure,

wherein the volume controlling unit determines the pressure of the touch input at the reference time as the reference pressure, and

wherein the reference time is a time point after the size of the pressure of the touch input is increased to a predetermined size or more.