Suggestion of visual effects based on detected sound patterns

• US 10,277,834 B2
• Filed: 01/10/2017
• Issued: 04/30/2019
• Est. Priority Date: 01/10/2017
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Greenberger, Jeremy A., Greenberger, Zachary M.
• Primary Examiner(s): Pullias, Jesse S
• Application Number:
  US15/402,271
• Publication Number:
  US 20180198990A1
• Time in Patent Office: 840 Days
• Field of Search: 704231-257, 704270-275
• CPC Class Codes: G10H 2210/076 for extraction of timing, tempo; Beat detection
• CPC Class Codes: G10L 15/26 Speech to text systems G10L15/08 takes precedence
• CPC Class Codes: G10L 21/10 Transforming into visible information
• CPC Class Codes: G10L 25/51 for comparison or discrimination
• CPC Class Codes: G10L 25/57 for processing of video signals
• CPC Class Codes: G10L 25/78 Detection of presence or absence of voice signals switching of direction of transmission by voice frequency in two-way loud-speaking telephone systems H04M9/10
• CPC Class Codes: H04N 5/2621 Cameras specially adapted for the electronic generation of special effects during image pickup, e.g. digital cameras, camcorders, video cameras having integrated special effects capability

Abstract

Disclosed embodiments provide techniques for suggesting a visual effect based on detected sounds. The sounds can be speech and/or music. Tempo and song identification techniques may be used to determine criteria for selecting visual effects to present to a user. The user selects a visual effect from the suggested visual effects and applies the visual effect to an image acquired by a camera. A modified image that combines the original acquired image with the visual effect is transmitted to another user during communication such as video chat, or, alternatively, the modified image may be posted to a social media account.

Claims

1. A computer-implemented method for providing a suggested visual effect based on a detected sound pattern, comprising:

receiving a first sound signal from a microphone coupled to a computing device;

detecting a sound pattern within the first sound signal;

selecting, based on the detected sound pattern, a visual effect from a plurality of visual effects in a visual effects library;

presenting the visual effect as a selectable option in a user interface on the computing device;

assigning, in response to the selection, a point to the visual effect in a database table;

receiving a second sound signal similar to the first sound signal; and

presenting the visual effect as a selectable option if the visual effect has a high point total as compared to point totals associated with other visual effects.

2. The method of claim 1, wherein detecting a sound pattern comprises detecting human speech.

3. The method of claim 2, further comprising:

performing a speech-to text-process on the human speech to derive a text string;

extracting keywords from the text string; and

wherein the selecting the visual effect from the plurality of visual effects in the visual effects library comprises performing a query of the visual effects library using the extracted keywords.

4. The method of claim 1, wherein detecting a sound pattern comprises detecting music.

5. The method of claim 4, further comprising:

performing a tempo detection process on the music to detect a tempo; and

wherein the selecting the visual effect from the plurality of visual effects in the visual effects library comprises performing a query of the visual effects library using the detected tempo.

6. The method of claim 4, further comprising:

performing a song detection process on the music to detect a song title; and

wherein selecting a visual effect from a plurality of visual effects in a visual effects library comprises performing a query of the visual effects library using metadata derived from the song title.

7. The method of claim 6, wherein the metadata includes the song title.

8. The method of claim 6, wherein the metadata includes an occasion category.

9. The method of claim 1, wherein selecting a visual effect comprises selecting an overlay.

10. The method of claim 1, wherein selecting a visual effect comprises selecting a facial feature modification.

11. A computer system comprising:

a processor;

a microphone;

a camera;

a memory coupled to the processor, the memory containing instructions, that when executed by the processor, perform the steps of;

receiving a first sound signal from a microphone coupled to a computing device;

detecting a sound pattern within the first sound signal;

selecting, based on the detected sound pattern, a visual effect from a plurality of visual effects in a visual effects library;

presenting the visual effect as a selectable option in a user interface on the computing device;

applying the visual effect to an image obtained by the camera upon selection of the presented visual effect;

assigning, in response to the selection, a point to the visual effect in a database table;

receiving a second sound signal similar to the first sound signal; and

presenting the visual effect as a selectable option if the visual effect has a high point total as compared to point totals associated with other visual effects.

12. The system of claim 11, wherein the memory further contains instructions, that when executed by the processor, perform the step of detecting human speech within the sound pattern.

13. The system of claim 12, wherein the memory further contains instructions, that when executed by the processor, perform the steps of:

performing a speech-to-text process on the human speech to derive a text string;

extracting keywords from the text string; and

wherein the selecting the visual effect from the plurality of visual effects in the visual effects library comprises performing a query of the visual effects library using the extracted keywords.

14. The system of claim 11, wherein the memory further contains instructions, that when executed by the processor, perform the step of detecting human music within the sound pattern.

15. The system of claim 14, wherein the memory further contains instructions, that when executed by the processor, perform the steps of:

performing a tempo detection process on the music to detect a tempo; and

wherein the selecting the visual effect from the plurality of visual effects in the visual effects library comprises performing a query of the visual effects library using the detected tempo.

16. The system of claim 14, wherein the memory further contains instructions, that when executed by the processor, perform the steps of:

performing a song detection process on the music to detect a song title; and

wherein the selecting a visual effect from the plurality of visual effects in the visual effects library cmprises performing a query of the visual effects library using metadata derived from the song title.

17. A computer program product for providing a suggested visual effect based on a detected sound pattern for an electronic computing device comprising a computer readable storage medium having program instructions embodied therewith, the program instructions executable by a processor to cause the electronic device to:

receive a first sound signal from a microphone coupled to a computing device;

detect a sound pattern within the first sound signal;

select, based on the detected sound pattern, a visual effect from a plurality of visual effects in a visual effects library;

present the visual effect as a selectable option in a user interface on the computing device;

assign, in response to the selection, a point to the visual effect in a database table;

receive a second sound signal similar to the first sound signal; and

present the visual effect as a selectable option if the visual effect has a high point total as compared to point totals associated with other visual effects.

18. The computer program product of claim 17, further comprising program instructions executable by the processor to cause the electronic device to:

detect human speech within the sound pattern;

perform a speech-to-text process on the human speech to derive a text string;

extract keywords from the text string; and

perform the selection of the visual effect from the plurality of visual effects in the visual effects library by performing a query of the visual effects library using the extracted keywords.

19. The computer program product of claim 17, further comprising program instructions executable by the processor to cause the electronic device to:

detect music within the sound pattern;

perform a tempo detection process on the music to detect a tempo; and

perform the selection of the visual effect from the plurality of visual effects in the visual effects library by performing a query of the visual effects library using the detected tempo.

20. The computer program product of claim 17, further comprising program instructions executable by the processor to cause the electronic device to:

detect music within the sound pattern;

perform a song detection process on the music to detect a song title; and

perform the selection of the visual effect from the plurality of visual effects in the visual effects library by performing a query of the visual effects library using metadata derived from the song title.