Method and system for communicating between a sender and a recipient via a personalized message including an audio clip extracted from a pre-existing recording

• US 10,200,323 B2
• Filed: 10/17/2017
• Issued: 02/05/2019
• Est. Priority Date: 06/30/2011
• Status: Active Grant

• Current Assignee: Audiobyte LLC
• Original Assignee: Audiobyte LLC
• Inventors: Guthery, Scott, van den Bosch, Richard, Vo, Brian, Leung, Nolan, Blacker, Andrew, van Suchtelen, John
• Primary Examiner(s): Washburn, Daniel C
• Assistant Examiner(s): Nguyen, Timothy
• Application Number:
  US15/786,119
• Publication Number:
  US 20180103000A1
• Time in Patent Office: 476 Days
• Field of Search: 704235
• CPC Class Codes: G06F 16/68 Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
• CPC Class Codes: G06F 16/685 using automatically derived transcript of audio data, e.g. lyrics speech recognition G10L15/00
• CPC Class Codes: G06F 3/04842 Selection of displayed objects or displayed text elements G06F3/0482 takes precedence
• CPC Class Codes: G06Q 20/26 Debit schemes, e.g. "pay now"
• CPC Class Codes: G06Q 20/3255 using mobile network messaging services for payment, e.g. SMS
• CPC Class Codes: G06Q 20/3272 using an audio code
• CPC Class Codes: G06Q 20/384 using social networks
• CPC Class Codes: G06Q 20/386 using messaging services or messaging apps using mobile network messaging services for payment, e.g. SMS G06Q20/3255
• CPC Class Codes: G06Q 30/0273 Determination of fees for advertising
• CPC Class Codes: G06Q 40/12 Accounting
• CPC Class Codes: G10L 15/26 Speech to text systems G10L15/08 takes precedence
• CPC Class Codes: H04L 51/04 Real-time or near real-time messaging, e.g. instant messaging [IM]
• CPC Class Codes: H04L 51/046 Interoperability with other network applications or services
• CPC Class Codes: H04L 51/10 Multimedia information
• CPC Class Codes: H04L 51/52 for supporting social networking services

Abstract

A method of communicating between a sender and a recipient via a personalized message, including steps of: (a) identifying text, via the user interface of a communication device, of a desired lyric phrase from within a pre-existing audio recording; (b) selecting visual data, such as an image or video, to be paired with the desired lyric phrase; (c) extracting audio substantially associated with the desired lyric phrase from the pre-existing recording into a desired audio clip; (d) inputting personalized text via the user interface; (e) creating the personalized message with the sender identification, the personalized text and access to the desired audio clip; and (e) sending an electronic message to the electronic address of the recipient. Clips may be generated automatically based on a relevance score. The electronic message may be a text message, instant message, or email message; this message may alternatively have a link to the personalized message.

Claims

1. A computer program product embodied on a non-transitory computer readable medium, comprising code executable by a computer having a processor and a graphical user interface and arranged to communicate with at least one repository of audio media and at least one repository of multimedia, the multimedia comprising at least one of static image files, dynamic images files, audio and video files, to cause the computer to carry out the following steps:

accessing the at least one repository of multimedia;

selecting, via the graphical user interface, at least one multimedia file to be retrieved, and retrieving, from the at least one repository of multimedia, the at least one multimedia file to be retrieved;

accessing the at least one repository of the audio media;

selecting, via the graphical user interface, at least one category of pre-existing recording to be retrieved;

retrieving, from the at least one repository of audio media, at least one pre-existing recording, said pre-existing recording associated with the at least one selected category;

selecting, via the graphical user interface, a desired phrase from within a pre-existing recording;

communicating the selection of the desired phrase to the at least one repository of audio media in the form of non-audio data identifying a phrase from an audio recording, the at least one audio file messaging software-as-a-service platform comprising an audio extraction engine that (i) receives the non-audio data identifying the phrase together with an instance of the audio recording, (ii) identifies a portion of the audio recording where the phrase is likely to be found at least in part by mapping each word in the phrase to one and only one vocal interval determined to exist in the audio recording, (iii) extracts the portion of the audio recording into a short snippet; and

(iv) writes the short snippet into a database;

amending the at least one multimedia file to include a text lyric, the text lyric corresponding to the desired phrase from the audio media repository; and

directing a file messaging software-as-a-service platform, via the graphical user interface, to generate and output the amended multimedia file as a message note for the intended recipient, wherein the amended multimedia file comprises the short snippet.

2. The computer program product of claim 1, wherein pre-existing recordings are sorted by emotion, and wherein the at least one category of pre-existing recording to be retrieved comprises data associated with a mood or emotion.

3. The computer program product of claim 2, wherein a phrase within a pre-existing recording is associated with a mood or emotion.

4. The computer program product of claim 1, wherein the set of first information further includes sender location information.

5. The computer program product of claim 4, wherein the sender location information is a geo-location.

6. The computer program product of claim 4, further comprising adjusting availability of pre-existing recordings to the sender as a result of the sender location information.

7. The computer program product of claim 1, wherein the audiovisual message note is at least one of:

an SMS message, an MMS message, an email message, a one-to-one communication over a social media service, or a post on the social media service.

8. The computer program product of claim 1, wherein the step of amending the at least one visual media file to include a text lyric further comprises editing at least one of a font of the text lyric or a color of the text lyric.

9. The computer program product of claim 1, further comprising applying at least one filter to the at least one visual media file.

10. The computer program product of claim 1, wherein the step of directing the file messaging software-as-a-service platform, via the graphical user interface, to generate and output an audiovisual message note for the intended recipient further comprises:

generating, from the at least one visual media file, the at least one text lyric, and user customization information, a preview image;

generating, and displaying on the graphical user interface, a preview screen, the preview screen comprising the preview image and an option to post the preview image.

11. The computer program product of claim 1, wherein the preview image further comprises the short snippet of the audio recording corresponding to the text lyric.

12. The computer program product of claim 1, further comprising:

receiving an advertisement from a sponsor to be displayed to the sender;

displaying the advertisement on the graphical user interface;

determining whether the sender has met an exposure threshold for exposure to the advertisement; and

debiting a cost associated with generating, outputting, and delivering an audiovisual message note from the sponsor.

13. The computer program product of claim 1, further comprising:

receiving an incentive offer from a sponsor to be displayed to the sender;

displaying the incentive offer on the graphical user interface;

reporting whether the sender has accepted the incentive offer or declined the incentive offer; and

if the sender has accepted the incentive offer, implementing the incentive offer.

14. The computer program product of claim 1, further comprising:

selecting a social network on which the audiovisual message note is to be shared; and

sharing the audiovisual message note on the social network.

15. The computer program product of claim 1, wherein the sending of messages requires a sender user account, and wherein the sender user account has a user credit balance comprising at least one of earned credits and paid-for credits.

16. The computer program product of claim 15, wherein at least one of the selection of a pre-existing recording by the sender or the selection of a phrase within the pre-existing recording by the sender causes the user credit balance of the sender to be debited.

17. The computer program product of claim 15, wherein the sender user account is associated with at least one purchased pre-existing recording, and wherein the selection of a purchased pre-existing recording by the sender or the selection of a phrase within the purchased pre-existing recording by the sender does not cause the user credit balance of the sender to be debited.

18. A multimedia messaging software-as-a-service platform, comprising:

a computer hardware infrastructure accessible at a network domain and comprising;

first program code that receives, from a sender, a set of first information, the set of first information including a message personalized for an intended recipient, together with non-audio data identifying an audio phrase from an audio recording, as well as a selection of at least one multimedia file; and

an audio extraction engine that (i) receives the non-audio data identifying the audio phrase together with an instance of the audio recording, (ii) identifies a portion of the audio recording where the audio phrase is likely to be found at least in part by mapping each word in the audio phrase to one and only vocal interval determined to exist in the audio recording, (iii) extracts the portion of the audio recording into a short snippet; and

(iv) writes the short snippet into a database; and

a message generator that combines a reference to the short snippet with the message and the at least one multimedia file to generate and output a multimedia message note for the intended recipient; and

second program code operative to cause delivery of the short snippet in response to receipt of data indicating that the reference is selected by the intended recipient, wherein the multimedia message comprises the short snippet.

19. A multimedia messaging software-as-a-service platform having a processor and a memory, the multimedia messaging software-as-a-service platform comprising:

first program code that receives, from a sender, a set of first information, the set of first information including a message personalized for an intended recipient, together with non-audio data identifying a phrase from an audio recording, and further receives a selection of at least one multimedia file;

an audio extraction engine that (i) receives the non-audio data identifying the phrase together with an instance of the audio recording, (ii) identifies a portion of the audio recording where the phrase is likely to be found at least in part by mapping each word in the phrase to exactly one vocal interval determined to exist in the audio recording, (iii) extracts the portion of the audio recording into a short snippet; and

(iv) writes the short snippet into a database;

a message generator that combines a reference to the short snippet with at least one multimedia file and the message to generate and output a multimedia message note for the intended recipient;

second program code operative to control access to the short snippet to a recipient of a multimedia message note having a user account, and prompting the intended recipient to log into or create a user account; and

third program code operative to cause delivery of the short snippet in response to receipt of data indicating that the reference is selected by the intended recipient, wherein the multimedia message comprises the short snippet.

20. The audiovisual messaging software-as-a-service platform of claim 19, wherein the audio extraction engine is further configured to generate an audio relevance score from the short snippet, the audio relevance score based on at least one of metadata of the audio recording, a tag associated with the audio recording, a tag associated with the short snippet, an internal popularity score of the audio recording, an internal popularity score of the short snippet, an external popularity score of the audio recording, or an external popularity score of the short snippet;

wherein the audiovisual messaging software-as-a-service platform is further configured to generate a visual media relevance score for the at least one visual media file, the visual media relevance score based on at least one of a tag relevance score associated with the visual media file and based on a comparison of a tag of the visual media file and a list of popular tags maintained by the audiovisual messaging software-as-a-service platform, an internal popularity score associated with the visual media file, and an external popularity score associated with the visual media file; and

wherein the message generator automatically combines the reference to the short snippet with the at least one visual media file based on the audio relevance score and the visual media relevance score, automatically generates the audiovisual message note, and automatically stores the audiovisual message note in a repository.