Continuous pitch-corrected vocal capture device cooperative with content server for backing track mix

• US 9,754,571 B2
• Filed: 05/22/2015
• Issued: 09/05/2017
• Est. Priority Date: 12/15/2009
• Status: Active Grant

• Current Assignee: Smule, Inc.
• Original Assignee: Smule, Inc.
• Inventors: Salazar, Spencer, Fiebrink, Rebecca A., Wang, Ge, Ljungstrom, Mattias, Smith, Jeffrey C., Cook, Perry R.
• Primary Examiner(s): He, Jialong
• Application Number:
  US14/720,320
• Publication Number:
  US 20160005416A1
• Time in Patent Office: 837 Days
• CPC Class Codes: G06F 3/165 Management of the audio stream, e.g. setting of volume, audio stream path
• CPC Class Codes: G10H 1/366 with means for modifying or correcting the external signal, e.g. pitch correction, reverberation, changing a singer's voice
• CPC Class Codes: G10H 1/368 displaying animated or moving pictures synchronized with the music or audio part
• CPC Class Codes: G10H 2210/251 Chorus, i.e. automatic generation of two or more extra voices added to the melody, e.g. by a chorus effect processor or multiple voice harmonizer, to produce a chorus or unison effect, wherein individual sounds from multiple sources with roughly the same timbre converge and are perceived as one
• CPC Class Codes: G10H 2210/331 Note pitch correction, i.e. modifying a note pitch or replacing it by the closest one in a given scale
• CPC Class Codes: G10H 2220/011 Lyrics displays, e.g. for karaoke applications
• CPC Class Codes: G10H 2240/125 Library distribution, i.e. distributing musical pieces from a central or master library
• CPC Class Codes: G10H 2240/175 for jam sessions or musical collaboration through a network, e.g. for composition, ensemble playing or repeating; Compensation of network or internet delays therefor
• CPC Class Codes: G10H 2240/211 Wireless transmission, e.g. of music parameters or control data by radio, infrared or ultrasound beam G10H2220/405
• CPC Class Codes: G10H 2240/251 Mobile telephone transmission, i.e. transmitting, accessing or controlling music data wirelessly via a wireless or mobile telephone receiver, analog or digital, e.g. DECT GSM, UMTS smartphone, PDA or palmtop used for musical purposes G10H2230/015; mobile ringtone G10H2230/021
• CPC Class Codes: G10L 21/013 Adapting to target pitch
• CPC Class Codes: G10L 21/0356 for synchronising with other signals, e.g. video signals

Abstract

Techniques have been developed to facilitate (1) the capture and pitch correction of vocal performances on handheld or other portable computing devices and (2) the mixing of such pitch-corrected vocal performances with backing tracks for audible rendering on targets that include such portable computing devices and as well as desktops, workstations, gaming stations, even telephony targets. Implementations of the described techniques employ signal processing techniques and allocations of system functionality that are suitable given the generally limited capabilities of such handheld or portable computing devices and that facilitate efficient encoding and communication of the pitch-corrected vocal performances (or precursors or derivatives thereof) via wireless and/or wired bandwidth-limited networks for rendering on portable computing devices or other targets.

Claims

1. A method comprising:

using a portable computing device for vocal performance capture, the portable computing device having a display, a microphone interface and a data communications interface;

retrieving via the data communications interface at least lyrics and timing information corresponding to a backing track;

at the portable computing device, audibly rendering a first encoding of the backing track and concurrently presenting the retrieved lyrics on the display in accord with the timing information;

at the portable computing device, capturing and pitch correcting a vocal performance of a user in accord with pitch correction settings; and

distorting the vocal performance capture in accord with an applied effect in correspondence with user interface gestures by the user, wherein a user interface gesture dynamically switches back and forth between melody and harmony note sets as operant pitch correction settings.

2. The method of claim 1,

storing an audio encoding of the pitch-corrected vocal performance to a local storage of the portable computing device.

3. The method of claim 1,

wherein the pitch correction settings include score-coded note targets.

4. The method of claim 3,

wherein a first user interface gesture switches off use of the score-coded note targets.

5. The method of claim 4,

wherein a second user interface gesture switches on use of the score-coded note targets.

6. The method of claim 1,

transmitting from the portable computing device to a remote server via the communications interface, an audio encoding of the pitch-corrected vocal performance for mixing with a second encoding of the backing track.

7. The method of claim 6,

wherein the second encoding is of higher quality or fidelity than the first encoding.

8. The method of claim 1,

wherein the pitch correction settings are specific to a particular artist.

17. The method of claim 1, wherein a second user interface gesture causes modification of the pitch correction settings.

9. A portable computing device comprising:

a display;

a microphone interface;

a data communications interface;

code executable on the portable computing device to retrieve via the data communications interface at least lyrics and timing information corresponding to a backing track;

code further executable on the portable computing device to audibly render a first encoding of the backing track and concurrently present the retrieved lyrics on the display in accord with the timing information;

code further executable on the portable computing device to capture and pitch correct a vocal performance of a user in accord with pitch correction settings; and

code further executable on the portable computing device to distort the vocal performance capture in accord with an applied effect in correspondence with user interface gestures by the user, wherein a user interface gesture dynamically switches back and forth between melody and harmony note sets as operant pitch correction settings.

10. The system of claim 9, further comprising:

code further executable on the portable computing device to store an audio encoding of the pitch-corrected vocal performance to a local storage of the portable computing device.

11. The system of claim 9,

wherein the pitch correction settings include score-coded note targets.

12. The system of claim 11,

wherein a first user interface gesture switches off use of the score-coded note targets.

13. The system of claim 12,

wherein a second user interface gesture switches on use of the score-coded note targets.

14. The system of claim 9,

code further executable on the portable computing device to transmit from the portable computing device to a remote server via the communications interface, an audio encoding of the pitch-corrected vocal performance for mixing with a second encoding of the backing track.

15. The system of claim 14,

wherein the second encoding is of higher quality or fidelity than the first encoding.

16. The system of claim 9, wherein the portable computing device is selected from the group of:

a mobile phone;

a personal digital assistant; and

a laptop computer, notebook computer, a pad-type computer or netbook.

18. The system of claim 9, wherein a second user interface gesture causes modification of the pitch correction settings.