Ambient Noise Based Augmentation of Media Playback

• US 20100146445A1
• Filed: 12/08/2008
• Published: 06/10/2010
• Est. Priority Date: 12/08/2008
• Status: Active Grant

• Current Assignee: Apple Inc.
• Original Assignee: Apple Inc.
• Inventors: Kraut, Joel
• Granted Patent: US 8,977,974 B2
• Time in Patent Office: Days
• US Class Current: 715/821
• CPC Class Codes: G06F 3/0484 for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range
• CPC Class Codes: G06F 3/162 Interface to dedicated audio devices, e.g. audio drivers, interface to CODECs
• CPC Class Codes: H03G 3/02 Manually-operated control H03G3/001 and H03G3/002 take precedence
• CPC Class Codes: H04M 1/72442 for playing music files
• CPC Class Codes: H04M 1/72478 for hearing-impaired users
• CPC Class Codes: H04N 21/4126 The peripheral being portable, e.g. PDAs or mobile phones
• CPC Class Codes: H04N 21/41407 embedded in a portable device, e.g. video client on a mobile phone, PDA, laptop constructional details of equipment or arrangements specially adapted for portable computer application G06F1/1626; arrangements specially adapted for mobile receivers in broadcast systems H04H20/57
• CPC Class Codes: H04N 21/42202 environmental sensors, e.g. for detecting temperature, luminosity, pressure, earthquakes
• CPC Class Codes: H04N 21/42203 sound input device, e.g. microphone
• CPC Class Codes: H04N 21/42224 Touch pad or touch panel provided on the remote control
• CPC Class Codes: H04N 21/439 Processing of audio elementary streams
• CPC Class Codes: H04N 21/4394 involving operations for analysing the audio stream, e.g. detecting features or characteristics in audio streams arrangements characterised by components specially adapted for monitoring, identification or recognition of audio in broadcast systems H04H60/58
• CPC Class Codes: H04N 21/4396 by muting the audio signal
• CPC Class Codes: H04N 21/4884 for displaying subtitles
• CPC Class Codes: H04N 5/60 for the sound signals
• CPC Class Codes: H04W 4/025 using location based information parameters

Abstract

Ambient noise sampled by a mobile device from a local environment is used to automatically trigger actions associated with content currently playing on the mobile device. In some implementations, subtitles or closed captions associated with the currently playing content are automatically invoked and displayed on a user interface based on a level of ambient noise. In some implementations, audio associated with the currently playing content is adjusted or muted. Actions can be automatically triggered based on a comparison of the sampled ambient noise, or an audio fingerprint of the sampled ambient noise, with reference data, such as a reference volume level or a reference audio fingerprint. In some implementations, a reference volume level can be learned on the mobile device based on ambient noise samples.

Claims

1. A computer-implemented method comprising:

presenting a user interface on a mobile device for displaying currently playing content;

sampling ambient noise from a local environment of the mobile device; and

performing one or more actions on the mobile device based on the ambient noise, where at least one action is associated with the content currently playing on the mobile device.

2. The method of claim 1, where performing one or more actions, comprises:

overlaying a partially transparent sheet on the user interface, the sheet including one or more options associated with the currently playing content.

3. The method of claim 2, further comprising:

overlaying one or more controls on the user interface which are operable through touch input to control the content.

4. The method of claim 1, where performing one or more actions, comprises:

automatically displaying subtitles associated with the currently playing content on the user interface.

5. The method of claim 1, where performing one or more actions, comprises:

automatically displaying closed captions associated with the currently playing content.

6. The method of claim 1, where performing one or more actions, comprises:

automatically pausing the currently playing content.

7. The method of claim 1, where performing one or more actions, comprises:

automatically adjusting a volume level of audio associated with the currently playing content.

8. The method of claim 1, where performing one or more actions, comprises:

automatically muting audio associated with the currently playing content.

9. The method of claim 1, further comprising:

converting the sample into an audio fingerprint;

comparing the audio fingerprint with reference data; and

determining an action to be performed on the mobile device based on results of the comparing, where the action is related to the currently playing content.

10. The method of claim 1, further comprising:

comparing the sample with a reference volume level; and

determining an action to be performed on the mobile device based on the comparing, where the action is related to the currently playing content.

11. The method of claim 10, further comprising:

automatically adjusting the reference volume level based on the samples.

12. The method of claim 1, further comprising:

obtaining input from one or more controls or sensors of the mobile device; and

determining an action to be performed on the mobile device based on the samples and the input, where the action is related to the currently playing content.

13. A computer-readable medium having instructions stored thereon, which, when executed by a processor, causes the processor to perform operations comprising:

presenting a user interface on a mobile device for displaying currently playing content;

sampling ambient noise from a local environment of the mobile device; and

performing one or more actions on the mobile device based on the ambient noise, where at least one action is associated with the content currently playing on the mobile device.

14. The computer-readable medium of claim 13, where performing one or more actions, comprises:

overlaying a partially transparent sheet on the user interface, the sheet including one or more options associated with the currently playing content.

15. The computer-readable medium method of claim 13, further comprising:

overlaying one or more controls on the user interface which are operable through touch input to control the content.

16. The computer-readable medium of claim 13, where performing one or more actions, comprises:

automatically displaying subtitles associated with the currently playing content on the user interface.

17. The computer-readable medium of claim 13, where performing one or more actions, comprises:

automatically displaying closed captions associated with the currently playing content.

18. The computer-readable medium of claim 13, where performing one or more actions, comprises:

automatically pausing the currently playing content.

19. The computer-readable medium of claim 13, where performing one or more actions, comprises:

automatically adjusting a volume level of audio associated with the currently playing content.

20. The computer-readable medium method of claim 13, where performing one or more actions, comprises:

automatically muting audio associated with the currently playing content.

21. The computer-readable medium of claim 13, further comprising:

converting the sample into an audio fingerprint;

comparing the audio fingerprint with reference data; and

determining an action to be performed on the mobile device based on results of the comparing, where the action is related to the currently playing content.

22. The computer-readable medium of claim 13, further comprising:

comparing the sample with a reference volume level; and

determining an action to be performed on the mobile device based on the comparing, where the action is related to the currently playing content.

23. The computer-readable medium of claim 13, further comprising:

obtaining input from one or more controls or sensors of the mobile device; and

determining an action to be performed on the mobile device based on the ambient noise and the input, where the action is related to the currently playing content.

24. A system comprising:

a processor; and

memory coupled to the processor and including instructions, which when executed by the professor, causes the processor to perform operations comprising;

presenting a user interface on a mobile device for displaying currently playing content;

sampling ambient noise from a local environment of the mobile device; and

performing one or more actions on the mobile device based on the ambient noise, where at least one action is associated with the content currently playing on the mobile device.