Image capturing device for setting one or more setting values for an imaging mechanism based on acquired sound data that includes information reflecting an imaging environment

• US 8,675,096 B2
• Filed: 03/29/2010
• Issued: 03/18/2014
• Est. Priority Date: 03/31/2009
• Status: Expired due to Fees

• Current Assignee: Panasonic Intellectual Property Corporation of America (Panasonic Holdings Corporation)
• Original Assignee: Panasonic Corporation (Panasonic Holdings Corporation)
• Inventors: Kawanishi, Ryouichi, Uenoyama, Tsutomu, Yamamoto, Yasutoshi, Miyazaki, Keiichi
• Primary Examiner(s): Velez, Roberto
• Assistant Examiner(s): BHUIYAN, FAYEZ A
• Application Number:
  US12/994,242
• Publication Number:
  US 20110069201A1
• Time in Patent Office: 1,450 Days
• Field of Search: 348/231.4
• US Class Current: 348/231.4
• CPC Class Codes: G03B 17/20 visible in viewfinder
• CPC Class Codes: G03B 31/00 Associated working of cameras or projectors with sound-recording or sound-reproducing means
• CPC Class Codes: G03B 7/091 Digital circuits
• CPC Class Codes: H04N 23/631 Graphical user interfaces [GUI] specially adapted for controlling image capture or setting capture parameters
• CPC Class Codes: H04N 23/633 for displaying additional information relating to control or operation of the camera

Abstract

An imaging device of the present invention comprises: an imaging mechanism; a sound acquisition unit operable to acquire sound data that includes information reflecting an imaging environment; and a setting unit operable to, based on the sound data acquired by the sound acquisition unit, select and set one or more setting values for controlling the imaging mechanism.

Claims

1. An imaging device comprising:

an imaging mechanism;

a sound acquisition unit operable to acquire sound data that includes information reflecting an imaging environment;

a setting unit operable to, based on the sound data acquired by the sound acquisition unit, select and set one or more setting values for controlling the imaging mechanism; and

a scene designation unit that (i) includes an extraction subunit operable to extract, from the sound data acquired by the sound acquisition unit, an environmental sound feature amount indicating the imaging environment, (ii) stores therein model information pieces that each defines various types of sound features, each of the types of sound features having a predetermined relationship with a corresponding one of imaging scenes, and (iii) is operable to designate one of the imaging scenes, with reference to the model information pieces and based on the environmental sound feature amount extracted by the extraction subunit, wherein

each feature amount is a vector quantity composed of, as components, one or more of attributes of sound that are represented by a power of sound, zero crossing, a spectrum-related feature amount, a cepstrum-related feature amount, and a chroma vector,

each model information piece includes information showing at least one value for each component of a vector quantity and is to be compared with each feature amount,

the scene designation unit designates the imaging scene based on likelihoods that respectively pertain to pairs of a model information piece and a feature amount, each likelihood indicating a matching degree between the vector quantity indicated by the corresponding model information piece and the vector quantity of the corresponding feature amount, and

the setting unit (i) stores therein correspondence information that associates each of the imaging scenes with one or more setting values, and (ii) selects and sets one or more setting values associated by the correspondence information with the imaging scene designated by the scene designation unit as the one or more setting values for controlling the imaging mechanism.

2. The imaging device of claim 1, wherein

the extraction subunit determines, over each of predetermined units of time, a feature amount indicating features of the sound data, and when a temporal change in the feature amounts determined over the predetermined units of time is less than a predetermined amount, extracts the feature amounts determined over the predetermined units of time as the environmental sound feature amount.

3. The imaging device of claim 2, wherein

each feature amount is a vector quantity composed of, as components, two or more of attributes of sound that are represented by a power of sound, zero crossing, a spectrum-related feature amount, a cepstrum-related feature amount, and a chroma vector,

each model information piece includes information showing at least one value for each component of a vector quantity and is to be compared with each feature amount, and

the scene designation unit designates the imaging scene based on likelihoods that respectively pertain to pairs of a model information piece and a feature amount, each likelihood indicating a matching degree between the vector quantity indicated by the corresponding model information piece and the vector quantity of the corresponding feature amount.

4. The imaging device of claim 3, wherein

the information included in each model information piece shows an average value and a variance value as the at least one value for each component of the vector quantity, and

the scene designation unit designates the imaging scene based on (i) likelihoods that respectively pertain to pairs of a model information piece and a feature amount, each likelihood indicating a matching degree between (a) the vector quantity that is indicated by the corresponding model information piece and that shows an average value and (b) the vector quantity of the corresponding feature amount, and (ii) variance values shown by the vector quantities indicated by the model information pieces.

5. The imaging device of claim 3, wherein

the scene designation unit designates the imaging scene based on, from among likelihoods that respectively pertain to pairs of a model information piece and a feature amount and that each indicate a matching degree between (i) the vector quantity of a corresponding one of the feature amounts which constitute the environmental sound feature amount and are determined over the predetermined units of time and (ii) the vector quantity indicated by the corresponding model information piece, one or more likelihoods excluding likelihoods that do not satisfy a criterion that, with regard to one of the model information pieces, each of the likelihoods is greater than a predetermined value for a predetermined number of consecutive units of time.

6. The imaging device of claim 3, further comprising

a correction unit that (i) stores therein information for associating each model information piece with processing for correcting the one or more setting values for controlling the imaging mechanism, (ii) designates one or more of the model information pieces each of whose likelihoods is greater than a predetermined value, and (iii) executes the processing associated with the designated one or more of the model information pieces.

7. The imaging device of claim 3, wherein

each model information piece belongs to one of a plurality of groups, and

the scene designation unit (i) stores therein each model information piece and a coefficient in association with each other, (ii) for each group, calculates a group likelihood based on the likelihood and the coefficient associated with each model information piece belonging to the group, and (iii) designates the imaging scene based on the group likelihoods.

8. The imaging device of claim 1, further comprising

an operation reception unit operable to receive an operation by a user, wherein

the scene designation unit (i) refers to each model information piece, (ii) selects two or more of the imaging scenes as candidates based on the environmental sound feature amount extracted by the extraction subunit, and (iii) designates one of the candidates as the imaging scene based on the selected candidates and the operation by the user received by the operation reception unit.

9. The imaging device of claim 1, wherein

the imaging mechanism includes an image data generation unit operable to generate image data by imaging, and

the setting unit selects and sets one or more setting values that are to be used for controlling the imaging mechanism after the imaging has been conducted for the image data, based on the sound data and the image data.

10. An integrated circuit comprising:

a sound acquisition circuit operable to acquire sound data that includes information reflecting an imaging environment;

a setting circuit operable to, based on the sound data acquired by the sound acquisition circuit, select and set one or more setting values for controlling image capturing; and

a scene designation circuit that (i) includes an extraction sub-circuit operable to extract, from the sound data acquired by the sound acquisition circuit, an environmental sound feature amount indicating the imaging environment, (ii) stores model information pieces that each defines various types of sound features, each of the types of sound features having a predetermined relationship with a corresponding one of imaging scenes, and (iii) is operable to designate one of the imaging scenes, with reference to the model information pieces and based on the environmental sound feature amount extracted by the extraction sub-circuit, wherein

each feature amount is a vector quantity composed of, as components, one or more of attributes of sound that are represented by a power of sound, zero crossing, a spectrum-related feature amount, a cepstrum-related feature amount, and a chroma vector,

each model information piece includes information showing at least one value for each component of a vector quantity and is to be compared with each feature amount,

the scene designation circuit designates the imaging scene based on likelihoods that respectively pertain to pairs of a model information piece and a feature amount, each likelihood indicating a matching degree between the vector quantity indicated by the corresponding model information piece and the vector quantity of the corresponding feature amount, and

the setting circuit (i) stores correspondence information that associates each of the imaging scenes with one or more setting values, and (ii) selects and sets one or more setting values associated by the correspondence information with the imaging scene designated by the scene designation circuit as the one or more setting values for controlling the image capturing.

11. An imaging method used by an imaging device that includes an imaging mechanism, the imaging method comprising:

a sound acquisition step of acquiring sound data that includes information reflecting an imaging environment;

a setting step of, based on the sound data acquired in the sound acquisition step, selecting and setting one or more setting values for controlling the imaging mechanism; and

a scene designation step that includes an extraction sub-step of extracting, from the sound data acquired in the sound acquisition step, an environmental sound feature amount indicating the imaging environment, the scene designation step also being a step of (i) storing model information pieces that each defines various types of sound features, each of the types of sound features having a predetermined relationship with a corresponding one of imaging scenes, and (iii) designating one of the imaging scenes, with reference to the model information pieces and based on the environmental sound feature amount extracted in the extraction sub-step, wherein

each feature amount is a vector quantity composed of, as components, one or more of attributes of sound that are represented by a power of sound, zero crossing, a spectrum-related feature amount, a cepstrum-related feature amount, and a chroma vector,

each model information piece includes information showing at least one value for each component of a vector quantity and is to be compared with each feature amount,

the scene designation step designates the imaging scene based on likelihoods that respectively pertain to pairs of a model information piece and a feature amount, each likelihood indicating a matching degree between the vector quantity indicated by the corresponding model information piece and the vector quantity of the corresponding feature amount, and

the setting step (i) stores correspondence information that associates each of the imaging scenes with one or more setting values, and (ii) selects and sets one or more setting values associated by the correspondence information with the imaging scene designated in the scene designation step as the one or more setting values for controlling the imaging mechanism.

12. A non-transitory recording medium having recorded thereon a program that causes a computer of an imaging device including an imaging mechanism to perform imaging processing, the imaging processing comprising:

a sound acquisition step of acquiring sound data that includes information reflecting an imaging environment;

a setting step of, based on the sound data acquired in the sound acquisition step, selecting and setting one or more setting values for controlling the imaging mechanism; and

a scene designation step that includes an extraction sub-step of extracting, from the sound data acquired in the sound acquisition step, an environmental sound feature amount indicating the imaging environment, the scene designation step also being a step of (i) storing model information pieces that each defines various types of sound features, each of the types of sound features having a predetermined relationship with a corresponding one of imaging scenes, and (iii) designating one of the imaging scenes, with reference to the model information pieces and based on the environmental sound feature amount extracted in the extraction sub-step, wherein

each feature amount is a vector quantity composed of, as components, one or more of attributes of sound that are represented by a power of sound, zero crossing, a spectrum-related feature amount, a cepstrum-related feature amount, and a chroma vector,

each model information piece includes information showing at least one value for each component of a vector quantity and is to be compared with each feature amount,

the scene designation step designates the imaging scene based on likelihoods that respectively pertain to pairs of a model information piece and a feature amount, each likelihood indicating a matching degree between the vector quantity indicated by the corresponding model information piece and the vector quantity of the corresponding feature amount, and

the setting step (i) stores correspondence information that associates each of the imaging scenes with one or more setting values, and (ii) selects and sets one or more setting values associated by the correspondence information with the imaging scene designated in the scene designation step as the one or more setting values for controlling the imaging mechanism.