User interfaces for virtual instruments

• US 9,928,817 B2
• Filed: 05/03/2017
• Issued: 03/27/2018
• Est. Priority Date: 05/16/2016
• Status: Active Grant

• Current Assignee: Apple Inc.
• Original Assignee: Apple Inc.
• Inventors: Little, Alexander H., Decker, Joshua W., Manjarrez, Eli T., Lane, Christopher C.
• Primary Examiner(s): FLETCHER, MARLON T
• Application Number:
  US15/585,478
• Publication Number:
  US 20170330539A1
• Time in Patent Office: 328 Days
• CPC Class Codes: G06F 3/0482 Interaction with lists of selectable items, e.g. menus
• CPC Class Codes: G06F 3/0488 using a touch-screen or digitiser, e.g. input of commands through traced gestures
• CPC Class Codes: G10H 1/0008 Associated control or indicating means
• CPC Class Codes: G10H 1/0066 using a MIDI interface
• CPC Class Codes: G10H 1/08 by combining tones G10H1/14, G10H1/16 take precedence; chord G10H1/38
• CPC Class Codes: G10H 1/386 One-finger or one-key chord systems
• CPC Class Codes: G10H 2210/201 Vibrato, i.e. rapid, repetitive and smooth variation of amplitude, pitch or timbre within a note or chord discontinuities, note sequences or separate note onsets during the variation G10H2210/161; tremolo, i.e. stepwise pitch alternation G10H2210/191
• CPC Class Codes: G10H 2210/225 Portamento, i.e. smooth continuously variable pitch-bend, without emphasis of each chromatic pitch during the pitch change, which only stops at the end of the pitch shift, as obtained, e.g. by a MIDI pitch wheel or trombone pitch bend with emphasis of each chromatic pitch during pitch change, e.g. glissando, G10H2210/221
• CPC Class Codes: G10H 2210/265 Acoustic effect simulation, i.e. volume, spatial, resonance or reverberation effects added to a musical sound, usually by appropriate filtering or delays physical modeling of room acoustics G10H2250/531; formant synthesis G10H2250/481
• CPC Class Codes: G10H 2210/381 Manual tempo setting or adjustment tempo setting by interpretation of conducting movements G10H2220/206
• CPC Class Codes: G10H 2220/096 using a touch screen touch screen note input, e.g. using a displayed keyboard G10H2220/241; personal digital assistant [PDA] G10H2230/015
• CPC Class Codes: G10H 2220/106 using icons, e.g. selecting, moving or linking icons, on-screen symbols, screen regions or segments representing musical elements or parameters
• CPC Class Codes: G10H 2220/241 on touchscreens, i.e. keys, frets, strings, tablature or staff displayed on a touchscreen display for note input purposes
• CPC Class Codes: G10H 2230/075 Spint stringed, i.e. mimicking stringed instrument features, electrophonic aspects of acoustic stringed musical instruments without keyboard; MIDI-like control therefor string instrument sound synthesis G10H2250/441
• CPC Class Codes: G10H 7/002 using a common processing for different operations or calculations, and a set of microinstructions (programme) to control the sequence thereof

Abstract

Embodiments of the present disclosure can provide systems, methods, and computer-readable medium for implementing user interfaces for interacting with a virtual instrument. For example, first touch input indicating a string location of a plurality of string locations within the note selection area. Audio output corresponding to the sting location may be presented on a speaker based at least in part on the first touch input. Second touch input corresponding to an ornamental interface element of the user interface may be received. In response to the first and second touch input, a series of two or more audio outputs may be presented on the speaker according to a predetermined pattern.

Claims

1. A method, comprising:

obtaining a plurality of audio samples corresponding to recordings of musical notes played by a physical instrument;

displaying, on a display screen of an electronic device, a user interface for a virtual instrument associated with the physical instrument, the user interface comprising a strum area, a variation control interface element, and a fret selection area that comprises a plurality of fret positions;

receiving, at the user interface of the electronic device, first touch input indicating a fret position of a plurality of fret positions within the fret selection area;

presenting, via a speaker of the electronic device, an audio sample of the plurality of audio samples based at least in part on the first touch input, the audio sample comprising a note corresponding to the fret position;

receiving, at the user interface, second touch input corresponding to a location within the variation control interface element, the second touch input being received while the first touch input is being received;

determining a speed at which to play the audio sample repeatedly based at least in part on the second touch input; and

repeatedly presenting, via the speaker of the electronic device, the audio sample based at least in part on the determined speed.

2. The method of claim 1, wherein the physical instrument comprises a pipa and the virtual instrument simulates a pipa.

3. The method of claim 1, wherein receiving the second touch input corresponding to the location within the variation control interface element causes the location to be associated with a default speed at which repeated notes are to be played.

4. The method of claim 3, further comprising:

receiving, at the user interface, a third touch input indicating a selection of an additional note selection option;

determining an additional audio sample of the plurality of audio samples based at least in part on the third touch input indicating the selection of the additional note selection option; and

repeatedly playing the additional audio sample based at least in part on the default speed.

5. The method of claim 3, further comprising:

receiving, at the user interface, one or more updates that a selected location within the variation control interface element has changed, the selected location being associated with a particular speed at which the audio sample is to be repeated; and

repeatedly presenting the audio sample according to the particular speed.

6. The method of claim 1, further comprising:

receiving, at the user interface, third touch input corresponding to the strum area; and

presenting a plurality of audio samples based at least in part on the fret position, the second touch input corresponding to the variation control interface element, and the third touch input corresponding to the strum area.

7. The method of claim 6, wherein providing the plurality of audio samples further comprises:

determining a subset of the plurality of audio samples based at least in part on the first touch input indicating the fret position and third touch input corresponding to the strum area;

presenting, via the speaker of the electronic device, the audio sample with the subset of the plurality of audio samples according to the third touch input; and

determining the first touch input indicating that the fret position is maintained, wherein the audio sample is repeatedly presented based at least in part on the determined speed in response to determining that the first touch input is maintained.

8. The method of claim 6, wherein at least two of the first touch input, the second touch input, or the third touch input are received at substantially a same time.

9. A system, comprising:

a memory configured to store computer-executable instructions;

a display screen;

a speaker; and

a processor in communication with the memory and the speaker, the processor configured to execute the computer-executable instructions to at least;

display, on the display screen, a user interface for a virtual instrument, the user interface comprising a variation control interface element and a fret selection area that comprises a plurality of fret positions;

receive, at the user interface, first touch input indicating a fret position of the plurality of fret positions within the fret selection area;

present, via the speaker, audio output based at least in part on the first touch input, the audio output comprising a note corresponding to the fret position;

receive, at the user interface, second touch input corresponding to the variation control interface element while the first touch input is being received; and

present, via the speaker, additional audio output that comprises a plurality of repeated notes based at least in part on the fret position and the second touch input corresponding to the variation control interface element.

10. The system of claim 9, wherein the fret position corresponds to a particular note frequency of a plurality of note frequencies that are playable by the virtual instrument.

11. The system of claim 9, wherein the processor is further configured to execute the computer-executable instructions to at least:

receive additional touch input at the variation control interface element corresponding to an amount of pressure applied to a touch interface associated with the variation control interface element; and

modify a speed at which the plurality of repeated notes is presented according to the additional touch input, wherein the speed at which the plurality of repeated notes are played increases as the amount of pressure applied to the touch interface is increased.

12. The system of claim 9, wherein the processor is further configured to execute the computer-executable instructions to at least:

present a selection interface accessible from the user interface, the selection interface providing one or more options corresponding to one or more musical scale modes, the one or more musical scale modes corresponding to different sets of musical notes that are playable by the virtual instrument;

receive, at the selection interface, an indication that a particular musical scale mode of the one or more musical scale modes is selected; and

associate the plurality of fret positions to a corresponding set of musical notes based at least in part on the musical scale mode selected.

13. The system of claim 9, wherein the processor is further configured to execute the computer-executable instructions to at least:

receive, at the user interface, third touch input associated with the fret selection area, the third touch input indicating a pitch bend request; and

modify the audio output based at least in part on the third touch input, the modification causing an increase in pitch of the audio output.

14. A computer-readable storage medium having stored thereon computer-executable instructions that, when executed by a processor, cause the processor to perform operations comprising:

displaying, on a display screen of an electronic device, a user interface for a virtual instrument associated with a physical instrument, the user interface comprising a plurality of note selection areas and a variation control interface element, the plurality of note selection areas being grouped into sets of note selection areas associated with a corresponding musical scale;

receiving, at the user interface, a first touch input indicating selection of a note selection area of a set of note selection areas corresponding to a particular musical scale;

determining an audio sample of a plurality of audio samples based at least in part on the first touch input indicating selection of the note selection area;

presenting, via a speaker of the electronic device, audio output corresponding to the audio sample;

receiving, at the user interface, a second touch input corresponding to a location within the variation control interface element;

determining a speed at which to present additional audio output based at least in part on the second touch input; and

presenting, via the speaker, the additional audio output that comprises a plurality of repeated notes based at least in part on the note selection area and the determined speed.

15. The computer-readable storage medium of claim 14, wherein the physical instrument comprises a pipa and the virtual instrument simulates a pipa.

16. The computer-readable storage medium of claim 14, having stored thereon additional computer-executable instructions that, when executed by a processor, cause the processor to perform operations comprising:

receiving a pitch bend request related to the note selection area, the pitch bend request comprising accelerometer data indicating a tilt angle of a device that is displaying the user interface;

modifying the audio output according to the tilt angle of the device; and

presenting, via the speaker, the modified audio output.

17. The computer-readable storage medium of claim 14, having stored thereon additional computer-executable instructions that, when executed by a processor, cause the processor to perform operations comprising:

determining that contact with the display screen at a display location corresponding to the first touch input has been maintained; and

presenting the audio output continuously based at least in part on the contact being maintained.

18. The computer-readable storage medium of claim 14, wherein repeatedly playing the audio output simulates a lunzhi picking technique.

19. The computer-readable storage medium of claim 14, wherein presenting the audio output comprising the plurality of repeated notes further comprises generating one or more modified audio outputs that vary in pitch or volume, wherein the one or more modified audio outputs are utilized with the audio output to simulate a lunzhi picking technique.

20. The computer-readable storage medium of claim 14, having stored thereon additional computer-executable instructions that, when executed by a processor, cause the processor to perform operations comprising:

displaying, on the display screen, a selection interface accessible from the user interface, the selection interface providing one or more options corresponding to plurality of auto-play options, the plurality of auto-play options corresponding to a plurality of dynamic musical arrangements, the dynamic musical arrangements differing from one another in at least a degree of complexity;

receiving an indication that a particular auto-play option is selected; and

presenting, on the speaker, a dynamic musical arrangement based at least in part on the auto-play option and the first touch input.

21. A method, comprising:

displaying, on a display screen of an electronic device, a user interface for a virtual instrument, the user interface comprising a plurality of string locations within a note selection area and a plurality of ornamental interface elements;

receiving, at the user interface of the electronic device, first touch input indicating a string location of a plurality of string locations within the note selection area;

presenting, on a speaker of the electronic device, audio output based at least in part on the first touch input, the audio output comprising a note corresponding to the string location;

receiving, at the user interface of the electronic device, second touch input corresponding to an ornamental interface element of the plurality of ornamental interface elements; and

presenting, on the speaker of the electronic device, a series of two or more audio outputs based at least in part on the audio output and the second touch input corresponding to the ornamental interface element, the series of two or more audio outputs being presented according to a predetermined pattern.

22. The method of claim 21, further comprising:

receiving, at the user interface of the electronic device, third touch input indicating a change of pressure that is being applied to a touch interface of the electronic device at the ornamental interface element; and

modifying a speed at which the series of the two or more audio outputs of the plurality of audio samples are presented based at least in part on the change of pressure.

23. The method of claim 22, wherein the speed is increased as the pressure is increased, and wherein the speed is decreased as the pressure is decreased.

24. The method of claim 23, wherein the two or more audio outputs comprise a first audio output associated with a first note and a second audio output associated with a second note, and where the first audio output and the second audio output differ at least by an intervallic distance between the first note and the second note.

25. The method of claim 24, wherein the intervallic distance comprises a whole step of a twelve-tone musical scale associated with the string location.

26. The method of claim 21, further comprising:

generating at least one audio output of the two or more audio outputs based at least in part on the note corresponding to the string location.

27. The method of claim 21, wherein the ornamental interface element comprises a grace-note control interface element, a trill control interface element, a vibrato control interface element, or a horse-effect selection interface element.

28. The method of claim 27, wherein the predetermined pattern comprises a first pattern when second touch input is received at the grace-note control interface element, a second pattern when the second touch input is received at the trill control interface element, a third pattern when the second touch input is received at the vibrato control interface element, and a fourth pattern when the second touch input is received at the horse-effect selection interface element.

29. A system, comprising:

a memory configured to store computer-executable instructions;

a display screen,

a speaker; and

a processor in communication with the memory and the speaker, the processor configured to execute the computer-executable instructions to at least;

displaying, on the display screen, a user interface for a virtual instrument, the user interface comprising a plurality of string locations within a note selection area and a plurality of ornamental interface elements comprising at least a vibrato control interface element;

receive, at the user interface, first touch input indicating a string location of the plurality of string locations within a note selection area;

present, via the speaker, audio output based at least in part on the first touch input, the audio output comprising a note corresponding to the string location;

receive, at the user interface, second touch input corresponding to selection of the vibrato control interface element;

modify the audio output based at least in part on the second touch input corresponding to the vibrato control interface element, the modified audio output and the audio output differing at least by pitch; and

present, via the speaker, the audio output and the modified audio output.

30. The system of claim 29, wherein the second touch input is received at the vibrato control interface element.

31. The system of claim 29, wherein the second touch input is initiated by a change in pressure applied to a touch interface of the display at the string location.

32. The system of claim 29, wherein the second touch input comprises a default vibrato control, and wherein subsequent presentation of audio output is based at least in part on subsequent string locations selections and the default vibrato control.

33. The system of claim 29, wherein the second touch input comprises a selection location within the vibrato control interface element, and wherein the audio output is modified further based at least in part on the selection location.

34. A computer-readable storage medium having stored thereon computer-executable instructions that, when executed by a processor, cause the processor to perform operations comprising:

obtaining a plurality of audio samples corresponding to recordings of musical notes played by a physical instrument;

displaying, on a display screen of an electronic device, a user interface for a virtual instrument associated with the physical instrument, the user interface comprising a plurality of note selection areas and vibrato control interface element, the plurality of note selection areas being grouped into sets of note selection areas individually associated with a corresponding musical scale;

receiving, at the user interface, a first touch input indicating selection of a note selection area of a set of note selection areas associated with a particular musical scale;

determining an audio output from the plurality of audio samples based at least in part on the first touch input indicating selection of the note selection area;

presenting, via a speaker of the electronic device, the audio output according to the first touch input;

receiving, at the user interface, a second touch input corresponding to the vibrato control interface element;

modifying a pitch of the audio output based at least in part on the second touch input corresponding to the vibrato control interface element.

35. The computer-readable storage medium of claim 34, wherein the second touch input comprises at least one of a location selection within the vibrato control interface element or a pressure applied to a touch interface corresponding to the vibrato control interface element.

36. The computer-readable storage medium of claim 34, having stored thereon additional computer-executable instructions that, when executed by a processor, cause the processor to perform operations comprising:

determining that the first touch input is being maintained; and

presenting the audio output continuously for as long as the first touch input is maintained.

37. The computer-readable storage medium of claim 34, wherein modifying the pitch comprises:

determining a plurality of frequencies that are within a threshold frequency distance of a musical note corresponding to the audio output; and

modifying the frequency of the audio output to the plurality of frequencies according to the second touch input.

38. The computer-readable storage medium of claim 37, wherein a speed at which the frequency of the audio output is modified is determined based at least in part on the second touch input.

39. The computer-readable storage medium of claim 34, wherein the physical instrument is an erhu and the virtual instrument simulates an erhu.

40. The computer-readable storage medium of claim 34, having stored thereon additional computer-executable instructions that, when executed by a processor, cause the processor to perform operations comprising:

displaying, on the display screen, a selection interface accessible from the user interface, the selection interface providing one or more options corresponding to plurality of auto-play options, the plurality of auto-play options corresponding to a plurality of dynamic musical arrangements, the dynamic musical arrangements differing from one another in at least a degree of complexity;

receiving an indication that a particular auto-play option is selected; and

presenting, via the speaker, a dynamic musical arrangement corresponding to the auto-play option selected.