SHOOTING DEVICE, SHOOTING METHOD, AND PROGRAM

• US 20180324358A1
• Filed: 11/11/2016
• Published: 11/08/2018
• Est. Priority Date: 11/26/2015
• Status: Active Grant

• Current Assignee: Sony Semiconductor Solutions Corporation (Sony Group Corp.)
• Original Assignee: Sony Semiconductor Solutions Corporation (Sony Group Corp.)
• Inventors: YAMADA, HIDESHI, ISO, DAISUKE, KURITA, TEPPEI, LI, SHUN
• Granted Patent: US 10,542,217 B2
• Time in Patent Office: Days
• CPC Class Codes: G03B 2207/005 involving control of motion blur
• CPC Class Codes: G03B 5/00 Adjustment of optical system relative to image or object surface other than for focusing
• CPC Class Codes: G03B 7/00 Control of exposure by setting shutters, diaphragms or filters, separately or conjointly control of exposure in television cameras by means of circuitry influencing the optical part of the camera H04N23/75
• CPC Class Codes: G06T 2207/20056 Discrete and fast Fourier transform, [DFT, FFT]
• CPC Class Codes: G06T 5/10 using non-spatial domain filtering
• CPC Class Codes: G06T 5/50 using two or more images, e.g. averaging or subtraction
• CPC Class Codes: G06T 5/70 Denoising; Smoothing
• CPC Class Codes: G06T 7/20 Analysis of motion motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51
• CPC Class Codes: H04N 23/60 Control of cameras or camera modules
• CPC Class Codes: H04N 23/6812 based on additional sensors, e.g. acceleration sensors
• CPC Class Codes: H04N 23/682 Vibration or motion blur correction
• CPC Class Codes: H04N 23/6845 by combination of a plurality of images sequentially taken
• CPC Class Codes: H04N 23/73 by influencing the exposure time
• CPC Class Codes: H04N 23/76 by influencing the image signals
• CPC Class Codes: H04N 23/81 for suppressing or minimising disturbance in the image signal generation

Abstract

The present disclosure relates to a shooting device, a shooting method, and a program capable of suppressing shooting of an image including large blurring due to motion of a camera such that an image with less blurring may be shot. The camera motion is detected, a distribution degree of a trajectory of the camera motion based on a camera motion detection result is calculated to be compared with a predetermined threshold, and start and finish of exposure is controlled on the basis of a comparison result. The present disclosure may be applied to the shooting device.

Claims

1. A shooting device comprising:

a camera motion detection unit which detects camera motion;

a comparison unit which calculates a distribution degree of a trajectory of the camera motion based on a result of the camera motion detection to compare with a predetermined threshold; and

an exposure control unit which controls start and finish of exposure on the basis of a comparison result of the comparison unit.

2. The shooting device according to claim 1,

wherein the comparison unit calculates a first principal component of covariance as the distribution degree of the camera motion trajectory based on the camera motion detection result to compare with the predetermined threshold.

3. The shooting device according to claim 1,

wherein the comparison unit generates a point spread function (PSF) image for the camera motion detection result as the distribution degree of the camera motion trajectory based on the camera motion detection result, and performs frequency analysis to compare with the predetermined threshold.

4. The shooting device according to claim 1,

wherein the comparison unit approximates detection results in a certain range retroactive to predetermined time from current time by a pronominal as the distribution degree of the camera motion trajectory based on the camera motion detection result, extrapolates (extrapolates) motion in future time from the polynomial to measure, and compares with the predetermined threshold.

5. The shooting device according to claim 1,

wherein the camera motion detection unit includes a gyroscope sensor, an acceleration sensor, a geomagnetic sensor, an altitude sensor, a vibration sensor, and a motion capture which tracks a marker of an object from a sub camera for detecting motion of a camera used for imaging different from the camera used for imaging to measure motion.

6. The shooting device according to claim 1,

wherein the exposure control unit controls to finish the exposure when it is regarded that the exposure is longer than minimum exposure time and not longer than maximum exposure time after the exposure is started and blurring is larger than predetermined blurring by the comparison result on the basis of the comparison result of the comparison unit.

7. The shooting device according to claim 1,

wherein the exposure control unit controls to finish the exposure when the maximum exposure time elapses after the exposure is started on the basis of the comparison result of the comparison unit.

8. The shooting device according to claim 1,

wherein, when the blurring is regarded to be larger than predetermined blurring by the comparison result at timing at which the exposure is started, the exposure control unit controls to delay the timing at which the exposure is started until the blurring is regarded to be smaller than the predetermined blurring by the comparison result on the basis of the comparison result of the comparison unit.

9. The shooting device according to claim 1,

wherein the exposure control unit controls to finish the exposure in consideration of a signal to noise ratio (SNR) on the basis of the comparison result of the comparison unit.

10. The shooting device according to claim 1, further comprising:

a noise removal unit which removes noise of an image by integrating a plurality of images taken at a predetermined interval by the camera.

11. The shooting device according to claim 10,

wherein the noise removal unit further includes a noise removal unit which integrates only an image in which the blurring is smaller than predetermined blurring out of a plurality of images taken by the camera and removes noise of the image.

12. The shooting device according to claim 10,

wherein the noise removal unit integrates a plurality of images taken by the camera while adding weights according to the exposure time, to remove the noise of the image.

13. The shooting device according to claim 10,

wherein the noise removal unit integrates a plurality of images taken by the camera while adding weights in consideration of a direction of blurring of each image, to remove the noise of the image.

14. The shooting device according to claim 10,

wherein the noise removal unit integrates a plurality of images taken by the camera while adding equal weight, to remove the noise of the image.

15. The shooting device according to claim 10,

wherein the noise removal unit applies fast Fourier transform (FFT) on a plurality of images taken by the camera, collects a component of predetermined amplitude for each frequency component, and applies inverse FFT to generate an image, so that the noise of the image is removed.

16. The shooting device according to claim 15,

wherein the noise removal unit applies the FFT on a plurality of images taken by the camera, collects a component of amplitude of a maximum value for each frequency component, and applies the inverse FFT to generate an image, so that the noise of the image is removed.

17. A shooting method comprising steps of:

detecting camera motion;

calculating a distribution degree of a trajectory of the camera motion on the basis of a result of the camera motion detection to compare with a predetermined threshold; and

controlling start and finish of exposure on the basis of a comparison result.

18. A program which allows a computer to serve as:

a camera motion detection unit which detects camera motion;

a comparison unit which calculates a distribution degree of a trajectory of the camera motion based on a result of the camera motion detection to compare with a predetermined threshold; and

an exposure control unit which controls start and finish of exposure on the basis of a comparison result of the comparison unit.