Method for processing sound by electronic device and electronic device thereof

• US 9,967,658 B2
• Filed: 06/20/2016
• Issued: 05/08/2018
• Est. Priority Date: 08/21/2015
• Status: Active Grant

• Current Assignee: Samsung Electronics Co. Ltd.
• Original Assignee: Samsung Electronics Co. Ltd.
• Inventors: Kim, Jun-Tai, Bae, Min-Ho, Choi, Hyun-Min, Ko, Han-Ho, Kim, Gang-Youl, Yang, Jae-Mo, Choi, Chul-Min, Son, Beak-Kwon, Lee, Nam-Il
• Primary Examiner(s): Diep, Trung
• Application Number:
  US15/187,271
• Publication Number:
  US 20170055072A1
• Time in Patent Office: 687 Days
• CPC Class Codes: G06F 3/01 Input arrangements or combined input and output arrangements for interaction between user and computer G06F3/16 takes precedence
• CPC Class Codes: G06V 40/166 using acquisition arrangements
• CPC Class Codes: H04N 1/2112 using still video cameras
• CPC Class Codes: H04N 23/61 based on recognised objects
• CPC Class Codes: H04N 23/611 where the recognised objects include parts of the human body
• CPC Class Codes: H04R 1/406 microphones
• CPC Class Codes: H04R 2400/01 Transducers used as a loudspeaker to generate sound aswell as a microphone to detect sound
• CPC Class Codes: H04R 2430/20 Processing of the output signals of the acoustic transducers of an array for obtaining a desired directivity characteristic H04R2203/12 takes precedence
• CPC Class Codes: H04R 2499/11 Transducers incorporated or for use in hand-held devices, e.g. mobile phones, PDA's, camera's
• CPC Class Codes: H04R 3/005 for combining the signals of two or more microphones specially adapted for hearing aids H04R25/407
• CPC Class Codes: H04R 5/027 Spatial or constructional arrangements of microphones, e.g. in dummy heads
• CPC Class Codes: H04S 2400/15 Aspects of sound capture and related signal processing for recording or reproduction

Abstract

An electronic device is provided. The electronic includes a camera module and a processor configured to capture at least one image of an object using the camera module, obtain a sound using a microphone operably connected to the processor when the at least one image is captured, determine whether the sound is related to the object, when the sound is determined to be unrelated to the object, change at least one attribute of the sound and store the changed at least one attribute of the sound.

Claims

1. An electronic device, comprising:

a camera module; and

a processor configured to capture at least one image of an object using the camera module, obtain a sound using a microphone operably connected to the processor when the at least one image is captured, determine whether the sound is related to the object, when the sound is determined to be unrelated to the object, change at least one attribute of the sound and store the changed at least one attribute of the sound.

2. The electronic device of claim 1, wherein the processor is further configured to store visual information which is related to the sound.

3. The electronic device of claim 1, further comprising a memory for storing information corresponding to one of the at least one image, information corresponding to the sound, and visual information related to the sound.

4. The electronic device of claim 1, wherein the processor is further configured to determine a direction of the sound, and when it is determined that the sound is a sound coming from a direction from a rear surface of the camera module, determine that the sound is unrelated to the object.

5. The electronic device of claim 4, wherein the processor is further configured to determine the direction of the sound using a direction of arrival (DOA) estimation technique.

6. The electronic device of claim 4, wherein the processor is further configured to, when the direction of the sound is a first direction, assign the sound to a first sound channel of multiple sound channels, and when the direction of the sound is a second direction, assign the sound to a second sound channel of the multiple sound channels.

7. The electronic device of claim 1, wherein the microphone is functional as one of a speaker and a receiver.

8. The electronic device of claim 1, wherein the processor is further configured to, when the sound is determined to be related to the object, not change the at least one attribute of the sound.

9. The electronic device of claim 1, further comprising at least one sensor for obtaining a distance from an object corresponding to the sound, and wherein the processor is further configured to determine whether the sound is related to the object based on a distance between the electronic device and the object.

10. The electronic device of claim 1, further comprising another camera module, wherein the processor is further configured to determine whether the at least one image obtained through the another camera module includes a face, when the at least one image obtained through the another camera module includes the face, recognize a mouth portion of the face, and determine whether the sound is related to the object based on whether the mouth portion moves.

11. A method for processing a sound by an electronic device, the method comprising:

capturing at least one image of an object using a camera operably connected to the electronic device;

obtaining a sound using at least one microphone operably connected to the electronic device when the at least one image is captured;

determining whether the sound is related to the object; and

when the sound is determined to be unrelated to the object, changing at least one attribute of the sound and storing the changed at least one attribute of the sound.

12. The method of claim 11, further comprising storing visual information which is related to the sound.

13. The method of claim 11, wherein determining whether the sound is related to the object includes determining a direction of the sound, and when the sound is coming from a direction from a rear surface of the camera, determining that the sound is unrelated to the object.

14. The method of claim 13, wherein determining the direction of the sound includes using a direction of arrival (DOA) estimation technique.

15. The method of claim 13, further comprising, when a direction of the sound is a first direction, assigning the sound to a first sound channel of multiple sound channels, and when the direction of the sound is a second direction, assigning the sound to a second sound channel of the multiple sound channels.

16. The method of claim 11, wherein the microphone is functional as one of a speaker and a receiver, and wherein the method further comprises activating at least one of the speaker function and the receiver function of the microphone upon capturing the at least one image.

17. The method of claim 11, wherein when the sound is determined to be related to the object, the at least one attribute of the sound is not changed.

18. The method of claim 11, wherein determining whether the sound is related to the object is based on a distance between the electronic device and the object.

19. The method of claim 11, further comprising:

determining whether at least one image obtained through another camera includes a face; and

when the at least one image obtained through the another camera includes the face, recognizing a mouth portion of the face, and wherein determining whether the sound is related to the object is based on whether the mouth portion moves.

20. A non-transitory computer-readable storage medium having thereon a program that executes a method for processing a sound by an electronic device, the method comprising:

capturing at least one image of an object using a camera operably connected to the electronic device;

obtaining a sound using at least one microphone operably connected to the electronic device when the at least one image is captured;

determining whether the sound is related to the object; and

when the sound is determined to be unrelated to the object, changing at least one attribute of the sound and storing the changed at least one attribute of the sound.