SOUND PICKUP DEVICE, SOUND PICKUP SYSTEM, SOUND PICKUP METHOD, PROGRAM, AND CALIBRATION METHOD

• US 20200137492A1
• Filed: 12/30/2019
• Published: 04/30/2020
• Est. Priority Date: 09/08/2017
• Status: Active Grant

• Current Assignee: Panasonic Intellectual Property Management Co., Ltd. (Panasonic Holdings Corporation)
• Original Assignee: Panasonic Intellectual Property Management Co., Ltd. (Panasonic Holdings Corporation)
• Inventors: HIROSE, YOSHIFUMI, ADACHI, YUSUKE
• Granted Patent: US 11,234,074 B2
• Time in Patent Office: Days
• CPC Class Codes: H04N 23/50 Constructional details
• CPC Class Codes: H04N 23/54 Mounting of pick-up tubes, electronic image sensors, deviation or focusing coils
• CPC Class Codes: H04N 23/55 Optical parts specially adapted for electronic image sensors; Mounting thereof
• CPC Class Codes: H04N 23/60 Control of cameras or camera modules
• CPC Class Codes: H04N 23/695 Control of camera direction for changing a field of view, e.g. pan, tilt or based on tracking of objects
• CPC Class Codes: H04N 23/698 for achieving an enlarged field of view, e.g. panoramic image capture
• CPC Class Codes: H04N 7/15 Conference systems
• CPC Class Codes: H04R 1/026 Supports for loudspeaker casings
• CPC Class Codes: H04R 1/028 associated with devices performing functions other than acoustics, e.g. electric candles
• CPC Class Codes: H04R 1/04 Structural association of microphone with electric circuitry therefor in deaf-aid sets H04R25/00
• CPC Class Codes: H04R 1/083 Special constructions of mouthpieces
• CPC Class Codes: H04R 1/406 microphones
• CPC Class Codes: H04R 2201/403 Linear arrays of transducers
• CPC Class Codes: H04R 2205/021 Aspects relating to docking-station type assemblies to obtain an acoustical effect, e.g. the type of connection to external loudspeakers or housings, frequency improvement
• CPC Class Codes: H04R 2430/20 Processing of the output signals of the acoustic transducers of an array for obtaining a desired directivity characteristic H04R2203/12 takes precedence
• CPC Class Codes: H04R 29/005 Microphone arrays
• CPC Class Codes: H04R 3/005 for combining the signals of two or more microphones specially adapted for hearing aids H04R25/407

Abstract

A sound pickup device includes a sound pickup unit, a mounting part, and a communication unit. The sound pickup unit includes a microphone array including a plurality of microphones. The sound pickup unit generates an acoustic signal that corresponds to ambient sound that has been picked up by the microphone array. An electronic device is mounted on the mounting part and the electronic device includes a camera that generates image data by imaging surroundings. The communication unit transmits the acoustic signal to the electronic device.

Claims

1. A sound pickup device comprising:

a sound pickup unit that includes a microphone array including a plurality of microphones, the sound pickup unit generating an acoustic signal that corresponds to ambient sound that has been picked up by the microphone array;

a mounting part on which an electronic device is mounted, the electronic device including a camera that generates image data by imaging surroundings; and

a communication unit that transmits the acoustic signal to the electronic device.

2. The sound pickup device according to claim 1, further comprising an optical system that images a region outside an angle of view of the camera of the electronic device.

3. The sound pickup device according to claim 2, further comprising a first movable part that causes a location of the optical system to be movable.

4. The sound pickup device according to claim 2, wherein

the optical system includes:

a lens that widens the angle of view of the camera; and

a prevention member that reflects or absorbs light traveling along an optical axis of the camera.

5. The sound pickup device according to claim 2, wherein

the optical system includes a conical mirror that causes the region outside the angle of view of the camera of the electronic device to be imaged.

6. The sound pickup device according to claim 1, wherein

the mounting part includes a fixing member that is movable to fix the electronic device.

7. The sound pickup device according to claim 6, wherein

the microphone array is attached to the fixing member.

8. The sound pickup device according to claim 6, wherein

the communication unit is attached to the fixing member.

9. A sound pickup system comprising:

the sound pickup device according to claim 1; and

the electronic device that is mounted on the sound pickup device.

10. The sound pickup system according to claim 9, wherein

the electronic device further includes a controller that determines a sound pickup direction based on the acoustic signal and the image data.

11. The sound pickup system according to claim 10, wherein

the controller associates location coordinates of the ambient sound picked up by the microphone array with location coordinates of the surroundings imaged by the camera based on a relative location from the camera to the microphone array.

12. The sound pickup system according to claim 10, wherein

the electronic device further includes a sensor that senses an optical-axis direction of the camera, and

the controller changes image-capturing setting conditions of the camera in accordance with the optical-axis direction of the camera that has been sensed by the sensor.

13. A sound pickup method for mounting an electronic device on a sound pickup device and picking up target sound,

the electronic device including a camera that generates image data by imaging surroundings,

the sound pickup device including:

a sound pickup unit that includes a microphone array including a plurality of microphones, the sound pickup unit generating an acoustic signal that corresponds to ambient sound that has been picked up by the microphone array;

a mounting part on which the electronic device is mounted; and

a communication unit that transmits the acoustic signal to the electronic device,

the sound pickup method comprising;

associating location coordinates of the ambient sound picked up by the microphone array with location coordinates of the surroundings imaged by the camera based on a relative location from the camera to the microphone array; and

determining a direction that the target sound will be picked up in based on the acoustic signal and the image data.

14. A non-transitory recording medium storing a program that causes a computer to perform the sound pickup method according to claim 13.

15. A calibration method that uses a sound pickup system, the sound pickup system including a display and a microphone array, the calibration method comprising:

displaying an image on the display;

outputting a location in the image;

picking up sound by using the microphone array;

estimating a location of the sound; and

associating coordinates of the location in the image with coordinates of the location of the sound.

16. The calibration method according to claim 15, wherein

in the outputting the location in the image, a mark is displayed in the location in the image on the display.