CONTROLLING MOBILE DEVICE BASED ON SOUND IDENTIFICATION

• US 20130201800A1
• Filed: 01/28/2013
• Published: 08/08/2013
• Est. Priority Date: 02/08/2012
• Status: Active Grant

• Current Assignee: Qualcomm, Inc.
• Original Assignee: Qualcomm, Inc.
• Inventors: Kim, Taesu, Hwang, Kyu Woong, Jin, Minho
• Granted Patent: US 9,524,638 B2
• Time in Patent Office: Days
• US Class Current: 367/197
• CPC Class Codes: G06F 21/35 communicating wirelessly
• CPC Class Codes: G06F 21/554 involving event detection and direct action
• CPC Class Codes: G06F 21/83 input devices, e.g. keyboards, mice or controllers thereof
• CPC Class Codes: G06F 2221/2101 Auditing as a secondary aspect
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: G08C 23/02 using infrasonic, sonic or ultrasonic waves
• CPC Class Codes: H04M 1/271 controlled by voice recognition
• CPC Class Codes: H04M 2250/74 with voice recognition means voice activated dialling H04M1/271; voice recognition algorithms G10L15/00
• CPC Class Codes: H04M 3/493 Interactive information services, e.g. directory enquiries ; Arrangements therefor, e.g. interactive voice response [IVR] systems or voice portals
• CPC Class Codes: H04W 4/00 Services specially adapted for wireless communication networks; Facilities therefor

Abstract

A method and apparatus for performing a function in a mobile device are disclosed. A media sound from a sound output device external to the mobile device is captured and a sound feature is extracted from the captured media sound. A function to be performed in the mobile device is determined by identifying at least one reference sound feature in a set of reference sound features based on the extracted sound feature, each reference sound feature in the set of reference sound features being associated with at least one of a plurality of media sounds and at least one of a plurality of functions. Further, the determined function is performed in the mobile device.

Claims

1. A method for performing a function in a mobile device, the method comprising:

capturing a media sound from a sound output device external to the mobile device;

extracting a sound feature from the captured media sound;

determining the function to be performed in the mobile device by identifying at least one reference sound feature in a set of reference sound features based on the extracted sound feature, each reference sound feature in the set of reference sound features being associated with at least one of a plurality of media sounds and at least one of a plurality of functions; and

performing the determined function in the mobile device.

2. The method of claim 1, wherein the at least one reference sound feature is associated with an advertisement, and wherein performing the determined function includes displaying the advertisement on the mobile device.

3. The method of claim 1, wherein the at least one reference sound feature is associated with a keyword to be searched by a search engine, and wherein performing the determined function includes:

transmitting the keyword to the search engine;

receiving a search result from the search engine; and

outputting the search result on the mobile device.

4. The method of claim 1, wherein performing the determined function includes adjusting a configuration of the mobile device.

5. The method of claim 1, wherein performing the determined function includes:

outputting a notification of the function to be performed;

receiving an input to perform the function; and

performing the function in response to the received input.

6. The method of claim 1, wherein extracting the sound feature from the captured media sound and determining the function to be performed by the mobile device are performed if a sound level of the captured media sound is greater than a predetermined sound level.

7. The method of claim 1, wherein extracting the sound feature from the captured media sound includes:

extracting an intermediate sound feature from the captured media sound; and

applying at least one basis filter to the intermediate sound feature to generate the sound feature.

8. The method of claim 7, wherein the at least one basis filter is selected from a plurality of basis filters based on a degree of similarity between sound features extracted from a pair of sample sounds by each of the plurality of basis filters.

9. The method of claim 1, wherein the at least one reference sound feature is associated with meta-data including the function to be performed on the mobile device and context information.

10. A method for performing a function in a mobile device, the method comprising:

capturing a composite sound including at least one media sound by a plurality of sound sensors;

separating the at least one media sound from the composite sound;

extracting at least one sound feature from the at least one separated media sound;

determining the function to be performed in the mobile device by identifying at least one reference sound feature in a set of reference sound features based on the at least one extracted sound feature, each reference sound feature in the set of reference sound features being associated with at least one of a plurality of media sounds and at least one of a plurality of functions; and

performing the determined function in the mobile device.

11. The method of claim 10, wherein the at least one reference sound feature is associated with an advertisement, and wherein performing the determined function includes displaying the advertisement on the mobile device.

12. The method of claim 10, wherein the at least one reference sound feature is associated with a keyword to be searched by a search engine, and wherein performing the determined function includes:

transmitting the keyword to the search engine;

receiving a search result from the search engine; and

outputting the search result on the mobile device.

13. The method of claim 10, wherein performing the determined function includes adjusting a configuration of the mobile device.

14. The method of claim 10, wherein performing the determined function includes:

outputting a notification of the function to be performed;

receiving an input to perform the function; and

performing the function in response to the received input.

15. The method of claim 10, wherein extracting the at least one sound feature from the at least one separated media sound includes:

selecting a media sound from the at least one separated media sound; and

extracting a sound feature from the selected media sound.

16. The method of claim 10, wherein extracting the at least one sound feature from the at least one separated media sound includes extracting sound features from all separated media sounds.

17. A server for determining a function to be performed by a mobile device, the server comprising:

a receiver configured to receive a sound feature, the sound feature being extracted from a media sound captured by the mobile device;

an identifying unit configured to identify at least one reference sound feature in a set of reference sound features based on the sound feature, each reference sound feature in the set of reference sound features being associated with at least one of a plurality of media sounds and at least one of a plurality of functions; and

a function determining unit configured to determine the function to be performed by the mobile device based on the at least one identified reference sound feature.

18. The server of claim 17, wherein the at least one reference sound feature is associated with an advertisement, and wherein the function determining unit is configured to determine the function as displaying the advertisement on the mobile device.

19. The server of claim 17, wherein the at least one reference sound feature is associated with a keyword to be searched by a search engine, and wherein the function determining unit is configured to determine the function as:

transmitting the keyword to the search engine;

receiving a search result from the search engine; and

outputting the search result on the mobile device.

20. The server of claim 17, wherein the function determining unit is configured to determine the function as adjusting a configuration of the mobile device.

21. The server of claim 17, wherein the function determining unit is configured to determine the function as:

outputting a notification of the function to be performed;

receiving an input to perform the function; and

performing the function in response to the received input.

22. The server of claim 17, wherein the sound feature is extracted from the captured media sound if a sound level of the captured media sound is greater than a predetermined sound level.

23. The server of claim 17, wherein the sound feature is extracted from the captured media sound by extracting an intermediate sound feature from the captured media sound and applying at least one basis filter to the intermediate sound feature to generate the sound feature.

24. The server of claim 23, further comprising a basis filter generator configured to select the at least one basis filter from a plurality of basis filters based on a degree of similarity between sound features extracted from a pair of sample sounds by each of the plurality of basis filters.

25. The server of claim 17, wherein the at least one reference sound feature is associated with meta-data including the function to be performed by the mobile device and context information.

26. A mobile device, comprising:

a sound sensor configured to capture a media sound from a sound output device external to the mobile device;

a sound feature extractor configured to extract a sound feature from the captured media sound;

an identifying unit configured to identify at least one reference sound feature in a set of reference sound features based on the sound feature, each reference sound feature in the set of reference sound features being associated with at least one of a plurality of media sounds and at least one of a plurality of functions;

a function determining unit configured to determine a function to be performed by the mobile device based on the at least one identified reference sound feature; and

a control unit configured to perform the determined function in the mobile device.

27. The mobile device of claim 26, wherein the at least one reference sound feature is associated with an advertisement, and wherein the control unit is configured to display the advertisement on the mobile device.

28. The mobile device of claim 26, wherein the at least one reference sound feature is associated with a keyword to be searched by a search engine, and wherein the control unit is configured to:

transmit the keyword to the search engine;

receive a search result from the search engine; and

output the search result on the mobile device.

29. The mobile device of claim 26, wherein the control unit is configured to adjust a configuration of the mobile device.

30. The mobile device of claim 26, wherein the control unit is configured to:

output a notification of the function to be performed;

receive an input to perform the function; and

perform the function in response to the received input.

31. The mobile device of claim 26, wherein the sound sensor is further configured to capture the media sound by receiving an environmental sound at a predetermined time interval and determining whether the received environmental sound is the media sound.

32. A non-transitory computer-readable storage medium storing instructions for performing a function in a mobile device, the instructions causing a processor to perform operations of:

capturing a media sound from a sound output device external to the mobile device;

extracting a sound feature from the captured media sound;

determining the function to be performed in the mobile device by identifying at least one reference sound feature in a set of reference sound features based on the extracted sound feature, each reference sound feature in the set of reference sound features being associated with at least one of a plurality of media sounds and at least one of a plurality of functions; and

performing the determined function in the mobile device.

33. The medium of claim 32, wherein the at least one reference sound feature is associated with an advertisement, and wherein performing the determined function includes displaying the advertisement on the mobile device.

34. The medium of claim 32, wherein the at least one reference sound feature is associated with a keyword to be searched by a search engine, and wherein performing the determined function includes:

transmitting the keyword to the search engine;

receiving a search result from the search engine; and

outputting the search result on the mobile device.

35. The medium of claim 32, wherein performing the determined function includes adjusting a configuration of the mobile device.

36. The medium of claim 32, wherein performing the determined function includes:

outputting a notification of the function to be performed;

receiving an input to perform the function; and

performing the function in response to the received input.

37. The medium of claim 32, wherein extracting the sound feature from the captured media sound and determining the function to be performed by the mobile device are performed if a sound level of the captured media sound is greater than a predetermined sound level.

38. The medium of claim 32, wherein extracting the sound feature from the captured media sound includes:

extracting an intermediate sound feature from the captured media sound; and

applying at least one basis filter to the intermediate sound feature to generate the sound feature.

39. The medium of claim 38, wherein the at least one basis filter is selected from a plurality of basis filters based on a degree of similarity between sound features extracted from a pair of sample sounds by each of the plurality of basis filters.

40. The medium of claim 32, wherein the at least one reference sound feature is associated with meta-data including the function to be performed in the mobile device and context information.

41. A server for determining a function to be performed by a mobile device, the server comprising:

means for receiving a sound feature, the sound feature being extracted from a media sound captured by the mobile device;

means for identifying at least one reference sound feature in a set of reference sound features based on the sound feature, each reference sound feature in the set of reference sound features being associated with at least one of a plurality of media sounds and at least one of a plurality of functions; and

means for determining the function to be performed by the mobile device based on the at least one identified reference sound feature.

42. The server of claim 41, wherein the at least one reference sound feature is associated with an advertisement, and wherein the means for determining is configured to determine the function as displaying the advertisement on the mobile device.

43. The server of claim 41, wherein the at least one reference sound feature is associated with a keyword to be searched by a search engine, and wherein the means for determining is configured to determine the function as:

transmitting the keyword to the search engine;

receiving a search result from the search engine; and

outputting the search result on the mobile device.

44. The server of claim 41, wherein the means for determining is configured to determine the function as adjusting a configuration of the mobile device.

45. The server of claim 41, wherein the means for determining is configured to determine the function as:

outputting a notification of the function to be performed;

receiving an input to perform the function; and

performing the function in response to the received input.

46. The server of claim 41, wherein the sound feature is extracted from the captured media sound if a sound level of the captured media sound is greater than a predetermined sound level.

47. The server of claim 41, wherein the sound feature is extracted from the captured media sound by extracting an intermediate sound feature from the captured media sound and applying at least one basis filter to the intermediate sound feature to generate the sound feature.

48. The server of claim 47, further comprising means for selecting the at least one basis filter from a plurality of basis filters based on a degree of similarity between sound features extracted from a pair of sample sounds by each of the plurality of basis filters.

49. The server of claim 41, wherein the at least one reference sound feature is associated with meta-data including the function to be performed by the mobile device and context information.

50. A mobile device, comprising:

means for capturing a media sound from a sound output device external to the mobile device;

means for extracting a sound feature from the captured media sound;

means for identifying at least one reference sound feature in a set of reference sound features based on the sound feature, each reference sound feature in the set of reference sound features being associated with at least one of a plurality of media sounds and at least one of a plurality of functions;

means for determining a function to be performed by the mobile device based on the at least one identified reference sound feature; and

means for performing the determined function in the mobile device.

51. The mobile device of claim 50, wherein the at least one reference sound feature is associated with an advertisement, and wherein the means for performing is configured to display the advertisement on the mobile device.

52. The mobile device of claim 50, wherein the at least one reference sound feature is associated with a keyword to be searched by a search engine, and wherein the means for performing is configured to:

transmit the keyword to the search engine;

receive a search result from the search engine; and

output the search result on the mobile device.

53. The mobile device of claim 50, wherein the means for performing is configured to adjust a configuration of the mobile device.

54. The mobile device of claim 50, wherein the means for performing is configured to:

output a notification of the function to be performed;

receive an input to perform the function; and

perform the function in response to the received input.