AUDIO ALTERATION TECHNIQUES

• US 20110058056A1
• Filed: 09/09/2009
• Published: 03/10/2011
• Est. Priority Date: 09/09/2009
• Status: Active Grant

• Current Assignee: Apple Inc.
• Original Assignee: Apple Inc.
• Inventors: Chiu, Kelvin, Lindahl, Aram
• Granted Patent: US 9,930,310 B2
• Time in Patent Office: Days
• US Class Current: 348/222.1
• CPC Class Codes: H04N 21/41407 embedded in a portable device, e.g. video client on a mobile phone, PDA, laptop constructional details of equipment or arrangements specially adapted for portable computer application G06F1/1626; arrangements specially adapted for mobile receivers in broadcast systems H04H20/57
• CPC Class Codes: H04N 21/42202 environmental sensors, e.g. for detecting temperature, luminosity, pressure, earthquakes
• CPC Class Codes: H04N 21/4325 by playing back content from the storage medium reproduction of recorded television signals H04N5/76; reproduction of recorded television signals H04N9/79
• CPC Class Codes: H04N 21/439 Processing of audio elementary streams
• CPC Class Codes: H04N 21/44008 involving operations for analysing video streams, e.g. detecting features or characteristics in the video stream arrangements characterised by components specially adapted for monitoring, identification or recognition of video in broadcast systems H04H60/59
• CPC Class Codes: H04N 21/440236 by media transcoding, e.g. video is transformed into a slideshow of still pictures, audio is converted into text
• CPC Class Codes: H04N 21/4524 involving the geographical location of the client retrieval from the Internet by querying based on geographical locations G06F16/9537; systems specially adapted for using geographical information in broadcast systems H04H60/70; protocols in which the network application is adapted for the location of the user terminal in communication control or processing H04L67/52; services making use of the location of users or terminals in wireless networks H04W4/02; locating users or terminals in wireless networks H04W64/00
• CPC Class Codes: H04N 21/47 End-user applications
• CPC Class Codes: H04N 21/8153 comprising still images, e.g. texture, background image
• CPC Class Codes: H04N 5/602 for digital sound signals
• CPC Class Codes: H04N 9/8063 using time division multiplex of the PCM audio and PCM video signals

Abstract

A method of altering audio output from an electronic device based on image data is provided. In one embodiment, the method includes acquiring image data and determining one or more characteristics of the image data. Such characteristics may include sharpness, brightness, motion, magnification, zoom setting, and so forth, as well as variation in any of the preceding characteristics. The method may also include producing audio output, wherein at least one characteristic of the audio output is determined based on one or more of the image data characteristics. Various audio output characteristics that may be varied based on the video data characteristics may include, for instance, pitch, reverberation, tempo, volume, filter frequency response, added sound effects, or the like. Additional methods, devices, and manufactures are also disclosed.

Claims

1. A method comprising:

receiving image data at an electronic device;

processing the image data via the electronic device, wherein processing the image data includes determining at least one image metric of the image data; and

outputting an audio signal from the electronic device, wherein the output audio signal includes at least one characteristic that varies based on the at least one image metric.

2. The method of claim 1, wherein processing the image data includes encoding video data via a video encoder of the electronic device and determining at least one image metric includes determining motion vectors for an object represented in the video data such that the output audio signal includes at least one characteristic that varies based on the motion vectors.

3. The method of claim 2, comprising varying at least one of a volume, a pitch, or a tempo of the audio signal based on the determined motion vectors.

4. The method of claim 1, wherein outputting the audio signal from the electronic device includes at least one of playing the audio signal to a user via a speaker or outputting electronic data encoding the audio signal to an additional electronic device.

5. The method of claim 1, comprising outputting video generated from the image data to a user via a display of the electronic device.

6. The method of claim 1, wherein receiving image data includes acquiring the image data via an image capture device of the electronic device.

7. An electronic device comprising:

an image capture device disposed in a housing;

an image signal processor disposed in the housing;

a video encoder disposed in the housing; and

audio processing logic disposed in the housing;

wherein the electronic device is configured to acquire video data via the image capture device, to determine a video characteristic of the video data via at least one of the image signal processor or the video encoder, to provide the determined video characteristic to the audio processing logic, and to generate an audio effect via the audio processing logic based on the determined video characteristic.

8. The electronic device of claim 7, wherein the electronic device is configured to determine motion of an object represented in the acquired video data via the video encoder and to generate the audio effect based on the motion of the object.

9. The electronic device of claim 8, wherein the audio effect includes at least one of variation of a pitch of an audio signal based on vertical motion of the object or variation of a volume of the audio signal based on horizontal motion of the object.

10. The electronic device of claim 8, wherein the audio effect includes changing the tempo of music played from the electronic device, wherein the changing of tempo is based on the motion of the object.

11. The electronic device of claim 7, wherein the electronic device is configured to encode the acquired video data via the video encoder and to output the generated audio effect to a user during encoding of the acquired video data.

12. The electronic device of claim 7, wherein the video encoder includes an H.264 video encoder.

13. A method comprising:

receiving image data at an electronic device;

analyzing image data via image processing logic of the electronic device; and

altering a characteristic of an audio signal based on the analysis of the image data.

14. The method of claim 13, wherein receiving the image data at the electronic device includes receiving a plurality of static images.

15. The method of claim 14, comprising sequentially displaying the plurality of static images and outputting audio to a user therewith, wherein the output audio varies based on the analysis of the plurality of static images via the image processing logic.

16. The method of claim 13, wherein analyzing the image data includes determining at least one of sharpness, brightness, or color statistics, and altering the characteristic of the audio signal includes altering the characteristic of the audio signal based on the at least one of sharpness, brightness, or color statistics.

17. The method of claim 13, wherein determining at least one of sharpness, brightness, or color statistics includes determining at least one of sharpness, brightness, or color statistics for a plurality of still images, for a sequence of frames in video data, or both.

18. An electronic system comprising:

a motion detecting device;

audio processing logic configured to output audio; and

a speaker configured to audibly provide the audio to a user;

wherein the audio processing logic is configured to alter a parameter of the audio based on motion data generated by the motion detecting device.

19. The electronic system of claim 18, comprising image processing circuitry configured to analyze image data and output image characteristics, wherein the audio processing logic is configured to alter the parameter or an additional parameter of the audio based on one or more image characteristics output by the image processing circuitry.

20. The electronic system of claim 19, wherein the image processing circuitry includes a processor configured to analyze the image data via execution of image analysis software.

21. The electronic system of claim 19, wherein the parameter of the audio that the audio processing logic is configured to alter includes at least four of:

pitch, tempo, timbre, volume, frequency range, a reverberation effect, a chorus effect, or a delay effect.

22. The electronic system of claim 18, wherein the motion data includes rotational data for the motion detecting device, such that the audio processing logic is configured to alter the parameter of the audio based on rotation of the motion detecting device.

23. The electronic system of claim 18, comprising a position detecting device, wherein the audio processing logic is configured to alter the parameter or an additional parameter of the audio based on position data generated by the position detecting device.

24. A manufacture comprising:

one or more tangible, computer-readable storage media having application instructions encoded thereon for execution by a processor, the application instructions comprising;

instructions for acquiring image data;

instructions for receiving an image attribute from image processing logic configured to encode the acquired image data; and

instructions for generating an audio effect, wherein the audio effect includes at least one audio characteristic that is altered based on the received image attribute.

25. The manufacture of claim 24, wherein the instructions for generating the audio effect include at least one of instructions for creating the audio effect via a synthesizer or instructions for altering a playback characteristic of a stored audio file.

26. The manufacture of claim 24, wherein the application instructions include instructions enabling a user to input preferences with respect to the generation of the audio effect.

27. The manufacture of claim 24, wherein the application instructions include:

instructions for providing a virtual record player application to a user via a display of an electronic device; and

instructions for generating an additional audio effect based on motion of the electronic device during operation of the virtual record player application.