Imaging apparatus, imaging method, reproducing apparatus and reproducing method, and recording medium for associating sound data with image data

• US 8,218,026 B2
• Filed: 05/21/2009
• Issued: 07/10/2012
• Est. Priority Date: 05/22/2008
• Status: Active Grant

• Current Assignee: Fujifilm Corporation (Fujifilm Holdings Corporation)
• Original Assignee: Fujifilm Corporation (Fujifilm Holdings Corporation)
• Inventors: Oishi, Makoto
• Primary Examiner(s): GILES, NICHOLAS G
• Application Number:
  US12/453,770
• Publication Number:
  US 20090295948A1
• Time in Patent Office: 1,146 Days
• Field of Search: 348/231.99, 348/231.2, 348/231.3, 348/231.4, 348/231.6, 348/222.1
• US Class Current: 348/222.1
• CPC Class Codes: H04N 1/2166 for mass storage, e.g. in document filing systems information retrieval G06F16/00
• CPC Class Codes: H04N 1/2195 with temporary storage before final recording or on play-back, e.g. in a frame buffer
• CPC Class Codes: H04N 2101/00 Still video cameras
• CPC Class Codes: H04N 2201/3264 of sound signals

Abstract

A sound is recorded when shooting a still image, an effect is applied on the still image in accordance with an analytical result of the recorded sound, and the sound and the effect-applied still image are recorded in association with each other, thereby allowing recording of the image having reality in addition to the sound. The still image and the sound are reproduced in association with each other, thereby allowing a viewer to feel an ambience and reality close to a memory that the photographer felt. Thus, image which easily reminds the situation and feeling during shooting can be recorded with a sound.

Claims

1. An imaging apparatus, comprising:

an imaging unit which converts an electric signal into image data and outputs the image data, the electric signal being acquired by an image pickup element photoelectrically converting light from an image of a subject formed through an imaging lens on a photosensitive surface of the image pickup element;

a sound acquiring unit which converts a sound which represents circumstances around the subject into sound data and outputs the sound data;

a sound analyzing unit which analyzes a feature of the sound based on the sound data outputted from the sound acquiring unit;

an effect application unit which applies an effect on the image data outputted from the imaging unit, the effect comprising an image processing in accordance with the feature of the sound analyzed by the sound analyzing unit; and

a recording unit which associates the image data generated by applying the effect by the effect application unit with the sound data outputted from the sound acquiring unit and records the image data and the sound data on a prescribed recording medium,

wherein the feature of the sound comprises a volume of the sound a duration of the sound, and a frequency of the sound, and

wherein each of a different combination of the volume of the sound, the duration of the sound, and the frequency of the sound causes the effect application unit to apply a different unique combination of hue, sharpness, and contrast as the image processing.

2. The imaging apparatus according to claim 1, wherein the effect includes image processing which changes at least one of a hue, a sharpness, a contrast, and a blurredness of the image.

3. The imaging apparatus according to claim 1, wherein the sound analyzing unit extracts the volume of the sound, the frequency of the sound, or a time-series variation thereof as the feature of the sound, compares the extracted feature of the sound with a previously-stored feature pattern, and applies the effect which is previously associated with a feature pattern matching the extracted feature of the sound.

4. The imaging apparatus according to claim 1, further comprising a sound acquisition control unit which controls a timing of a start, a stop of a conversion and the output of the sound data by the sound acquiring unit in accordance with the feature of the sound analyzed by the sound analyzing unit.

5. The imaging apparatus according to claim 4, wherein the sound acquisition control unit starts outputting the sound data when detecting that the volume of the sound becomes larger than or equal to a prescribed value, terminates outputting the sound data when outputting the sound data over a prescribed first time period, or when the volume of the sound falls equal to or below the prescribed value and the falling continues over a prescribed second time period.

6. The imaging apparatus according to claim 1, further comprising a moving image creating unit which creates image frames of a moving image by applying a plurality of different effects on the image data outputted from the imaging unit, and creates moving image data based on the image frames,

wherein the recording unit associates the moving image data created by the moving image creating unit and the sound data with each other and records the associated data.

7. The imaging apparatus according to claim 1, further comprising a continuous shooting control unit which controls the imaging unit to output a prescribed count of the image data at prescribed time intervals,

wherein the effect application unit executes image processing on the prescribed count of image data outputted from the imaging unit using the image data and another image data captured before and/or after the image data in accordance with a control of the continuous control unit.

8. The imaging apparatus according to claim 1, further comprising a sound editing unit which edits a content of the sound data from the sound acquiring unit in accordance with the feature of the sound analyzed by the sound analyzing unit.

9. The imaging apparatus according to claim 8, wherein the sound editing unit performs at least one of extracting a specific frequency from the sound data, eliminating noise, extracting a sound of a specific person, and cutting off sound data whose sound volume does not reach a prescribed sound volume.

10. The imaging apparatus according to claim 1, further comprising a detection unit which detects a physical status of the imaging apparatus,

wherein the effect application unit applies the effect in accordance with the physical status detected by the detection unit.

11. The imaging apparatus according to claim 10, wherein the effect application unit applies a specific effect in accordance with a specific physical status detected by the detection unit.

12. The imaging apparatus according to claim 1, wherein the effect application unit changes at least one of a magnification ratio, a zooming speed, an amount of shift, a shift rate, a turning ratio, and a turning speed in a processing of the effect on the image in accordance with a variation of the volume of the sound or a rate of variation of the sound.

13. The imaging apparatus according to claim 1, wherein the sound analyzing unit extracts the feature of the sound, compares the extracted feature of the sound with a previously-stored feature pattern, and applies the effect which is previously associated with a feature pattern that matches the extracted feature of the sound.

14. The imaging apparatus according to claim 1, further comprising a sound editing unit which performs at least one of extracting a specific frequency from the sound data, eliminating noise, and extracting a sound of a specific person to edit a content of the sound data from the sound acquiring unit in accordance with the feature of the sound analyzed by the sound analyzing unit.

15. The imaging apparatus according to claim 1, wherein said each of the different combination of the volume of the sound, the duration of the sound, and the frequency of the sound causes a different blurredness for the image.

16. A reproducing apparatus reproducing image data and sound data in association with each other, the image data and sound data being recorded on a prescribed recording medium by the imaging apparatus according to claim 1.

17. An imaging method, comprising:

converting an electric signal into image data and outputting the image data, the electric signal being acquired by an image pickup element photoelectrically converting light from an image of a subject formed through an imaging lens on a photosensitive surface of the image pickup element;

converting a sound which represents circumstances around the subject into sound data and outputting the sound data;

analyzing a feature of the sound based on the outputted sound data;

applying an effect on the outputted image data, the effect comprising an image processing in accordance with the analyzed feature of the sound; and

associating the image data generated by applying the effect with the outputted sound data and recording the image data and the sound data on a prescribed recording medium,

wherein the feature of the sound comprises a volume of the sound, a duration of the sound, and a frequency of the sound, and

wherein each of a different combination of the volume of the sound, the duration of the sound, and the frequency apply different unique combination of hue, sharpness, and contrast as the image processing.

18. A reproducing method reproducing image data and sound data in association with each other, the image data and sound data being recorded on a prescribed recording medium by the imaging method according to claim 17.

19. A non-transitory computer-readable storage medium encoded with a computer program, the program comprising computer-executable instructions of:

acquiring image data and sound data recorded concurrently with the image data said sound data being converted from a sound which represents circumstances around a subject of the image data;

analyzing a feature of the sound based on the sound data;

applying an effect on the image data, the effect comprising an, image processing in accordance with the analyzed feature of the sound; and

reproducing image data generated by applying the effect and the sound data in association with each other,

wherein the feature of the sound comprises a volume of the sound, a duration of the sound, and a frequency of the sound, and

wherein each of a different combination of the volume of the sound, the duration of the sound, and the frequency of the sound causes the applying to apply a different unique combination of hue, sharpness, and contrast as the image processing.