Controlling mobile device based on sound identification

• US 9,524,638 B2
• Filed: 01/28/2013
• Issued: 12/20/2016
• Est. Priority Date: 02/08/2012
• Status: Active Grant

• Current Assignee: Qualcomm, Inc.
• Original Assignee: Qualcomm, Inc.
• Inventors: Kim, Taesu, Hwang, Kyu Woong, Jin, Minho
• Primary Examiner(s): BROWN, VERNAL U
• Application Number:
  US13/752,080
• Publication Number:
  US 20130201800A1
• Time in Patent Office: 1,422 Days
• Field of Search: 340/12.22, 340/5.61, 340/4, 340/11, 340/4.4, 704/233
• US Class Current: 1/1
• CPC Class Codes: G06F 21/35 communicating wirelessly
• CPC Class Codes: G06F 21/554 involving event detection and direct action
• CPC Class Codes: G06F 21/83 input devices, e.g. keyboards, mice or controllers thereof
• CPC Class Codes: G06F 2221/2101 Auditing as a secondary aspect
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: G08C 23/02 using infrasonic, sonic or ultrasonic waves
• CPC Class Codes: H04M 1/271 controlled by voice recognition
• CPC Class Codes: H04M 2250/74 with voice recognition means voice activated dialling H04M1/271; voice recognition algorithms G10L15/00
• CPC Class Codes: H04M 3/493 Interactive information services, e.g. directory enquiries ; Arrangements therefor, e.g. interactive voice response [IVR] systems or voice portals
• CPC Class Codes: H04W 4/00 Services specially adapted for wireless communication networks; Facilities therefor

Abstract

A method and apparatus for performing a function in a mobile device are disclosed. A media sound from a sound output device external to the mobile device is captured and a sound feature is extracted from the captured media sound. A function to be performed in the mobile device is determined by identifying at least one reference sound feature in a set of reference sound features based on the extracted sound feature, each reference sound feature in the set of reference sound features being associated with at least one of a plurality of media sounds and at least one of a plurality of functions. Further, the determined function is performed in the mobile device.

Claims

1. A method for performing an operation at a mobile device, the method comprising:

capturing a sound from a sound output device external to the mobile device;

determining confidence values for a plurality of basis filters;

selecting at least one basis filter of the plurality of basis filters based on the confidence values;

processing, based on the at least one basis filter, data representing the sound to extract a sound feature from the data, the sound feature associated with the sound; and

performing the operation of the mobile device responsive to the sound feature.

2. The method of claim 1, further comprising:

comparing the sound feature to a plurality of reference sound features, each of the plurality of reference sound features being associated with the operation; and

identifying a first reference sound feature of the plurality of reference sound features based on the sound feature, the first reference sound feature associated with the operation.

3. The method of claim 1, wherein processing the data representing the sound is performed in response to a determination that a sound level of the sound is greater than a threshold sound level.

4. The method of claim 1, wherein processing the data representing the sound comprises:

extracting an intermediate sound feature from the data representing the sound, the intermediate sound feature comprising a power spectrum associated with the sound feature; and

applying the at least one basis filter to the intermediate sound feature to generate the sound feature.

5. The method of claim 1, wherein the at least one basis filter is selected from the plurality of basis filters further based on a plurality of sample sounds extracted from a sample sound database.

6. The method of claim 5, wherein the at least one basis filter is selected to reduce differences between a reference sound of the plurality of sample sounds and a distorted reference sound of the plurality of sample sounds.

7. The method of claim 5, wherein the plurality of sample sounds from the sample sound database includes three samples sounds.

8. The method of claim 5, wherein the confidence values are indicative of an amount of similarity between sound features extracted from a pair of sample sounds of the plurality of sample sounds.

9. The method of claim 1, wherein the sound feature is associated with meta-data including a volume function and context information.

10. The method of claim 1, wherein the sound is captured in a first mode of the mobile device and the data representing the sound is processed in a second mode of the mobile device, wherein the first mode is a lower power mode than the second mode, and wherein the second mode corresponds to a portion of a duty cycle of the mobile device.

11. The method of claim 1, wherein the sound feature corresponds to an announcement associated with a vehicle, and wherein the operation corresponds to providing an output via an interface of the mobile device, the output based on the announcement.

12. The method of claim 1, wherein the sound feature corresponds to a command to reduce a volume of the mobile device, and wherein the operation corresponds to reducing the volume of the mobile device.

13. The method of claim 1, wherein the mobile device comprises a mobile communication device.

14. A method for performing an operation at a mobile device, the method comprising:

capturing, by a plurality of sound sensors, a composite sound including at least one media sound;

determining confidence values for a plurality of basis filters;

selecting at least one basis filter of the plurality of basis filters based on the confidence values;

processing, based on the at least one basis filter, data representing the composite sound to extract sound data representing the at least one media sound from the data representing the composite sound and to extract at least one sound feature from the sound data representing the at least one media sound; and

performing the operation of the mobile device responsive to the at least one sound feature.

15. The method of claim 14, further comprising:

comparing the at least one sound feature to a plurality of reference sound features, each of the plurality of reference sound features associated with the operation; and

identifying a first reference sound feature of the plurality of reference sound features based on the at least one sound feature, the first reference sound feature associated with the operation.

16. The method of claim 14, wherein processing the data representing the composite sound includes:

selecting media sound data representing a particular media sound from the sound data representing the at least one media sound; and

extracting a sound feature from the selected media sound data representing the particular media sound.

17. The method of claim 14, wherein extracting the at least one sound feature from the sound data representing the at least one media sound includes extracting sound features from composite sound data representing all media sounds embedded within the composite sound.

18. The method of claim 14, wherein the mobile device comprises a mobile communication device.

19. A mobile device, comprising:

a sound sensor configured to capture a sound from a sound output device external to the mobile device;

a basis filter generator configured to determine confidence values for a plurality of basis filters and to select at least one basis filter of the plurality of basis filters based on the confidence values;

a sound feature extractor configured to process, based on the at least one basis filter, data representing the sound by extracting a sound feature from the data representing the sound after the sound sensor captures the sound; and

a control unit configured to perform an operation of the mobile device responsive to the sound feature.

20. The mobile device of claim 19, further comprising an identifying unit configured to identify a first reference sound feature from a plurality of reference sound features based on the sound feature, the first reference sound feature associated with the operation, wherein the control unit is configured to perform the operation based on the identification of the first reference sound feature.

21. The mobile device of claim 19, wherein the mobile device comprises a mobile communication device.

22. The mobile device of claim 19, further comprising:

an antenna; and

a receiver coupled to the antenna and configured to receive an encoded audio signal from a base station.

23. The mobile device of claim 22, wherein the receiver is configured to digitize the received signal.

24. The mobile device of claim 19, further comprising:

an antenna; and

a transmitter coupled to the antenna and configured to generate a modulated signal associated with the sound and to provide the signal to the antenna.

25. The mobile device of claim 24, further comprising a processor coupled to the transmitter, wherein the processor includes the sound feature extractor.

26. The mobile device of claim 25, wherein the antenna, the transmitter, and the processor are integrated into a mobile communication device.

27. A non-transitory computer-readable storage medium storing instructions for performing an operation at a mobile device, the instructions causing a processor to perform operations comprising:

capturing a sound from a sound output device external to the mobile device;

determining confidence values for a plurality of basis filters;

selecting at least one basis filter of the plurality of basis filters based on the confidence values;

after capturing the sound, processing, based on the at least one basis filter, data representing the sound to extract a sound feature from the data, the sound feature representing the sound; and

performing the operation of the mobile device responsive to the sound feature.

28. The medium of claim 27, wherein the operations further comprise:

comparing the sound feature to a plurality of reference sound features, each of the plurality of reference sound features being associated with the operation; and

identifying a first reference sound feature of the plurality of reference sound features based on the sound feature, the first reference sound feature associated with the operation.

29. The medium of claim 27, wherein processing the data representing the sound is performed in response to a determination that the sound comprises a media sound.

30. The medium of claim 27, wherein processing the data representing the sound comprises:

extracting an intermediate sound feature from the data representing the sound, the intermediate sound feature comprising a power spectrum associated with the sound feature; and

applying the at least one basis filter to the intermediate sound feature to generate the sound feature.

31. The medium of claim 30, wherein the at least one basis filter is selected from the plurality of basis filters further based on a plurality of sample sounds extracted from a sample sound database.

32. The medium of claim 27, wherein the sound feature is associated with meta-data including a volume function and context information.

33. An apparatus comprising:

means for capturing a sound from a sound output device external to a mobile device;

means for determining confidence values for a plurality of basis filters;

means for selecting at least one basis filter of the plurality of basis filters based on the confidence values;

means for processing, based on the at least one basis filter, data representing the sound to extract a sound feature from the data after the means for capturing the sound has captured the sound, the data representing the sound feature; and

means for performing an operation of the mobile device responsive to the sound feature.

34. The apparatus of claim 33, further comprising:

means for identifying a first reference sound feature from a plurality of reference sound features based on the sound feature, the first reference sound feature associated with an operation, wherein the means for performing the operation is configured to perform the operation based on the identification of the first reference sound feature.

35. The apparatus of claim 33, wherein the means for capturing the sound from the sound output device, the means for processing data representing the sound, and the means for performing the operation responsive to the sound feature are integrated into a mobile communicate device.