IMAGING SYSTEM AND IMAGING METHOD

• US 20190312080A1
• Filed: 06/20/2019
• Published: 10/10/2019
• Est. Priority Date: 02/28/2017
• Status: Active Grant

• Current Assignee: Panasonic Intellectual Property Management Co., Ltd. (Panasonic Holdings Corporation)
• Original Assignee: Panasonic Intellectual Property Management Co., Ltd. (Panasonic Holdings Corporation)
• Inventors: NOZAWA, KATSUYA
• Granted Patent: US 10,903,264 B2
• Time in Patent Office: Days
• CPC Class Codes: G06T 7/73 using feature-based methods
• CPC Class Codes: H01L 27/142 Energy conversion devices photovoltaic modules or arrays of single photovoltaic cells comprising bypass diodes integrated or directly associated with the devices H01L31/0443; photovoltaic modules composed of a plurality of thin film solar cells deposited on the same substrate H01L31/046
• CPC Class Codes: H01L 27/14612 involving a transistor
• CPC Class Codes: H01L 27/14623 Optical shielding
• CPC Class Codes: H01L 27/14665 Imagers using a photoconductor layer
• CPC Class Codes: H01L 27/14812 Special geometry or disposition of pixel-elements, address lines or gate-electrodes
• CPC Class Codes: H04N 23/00 Cameras or camera modules comprising electronic image sensors; Control thereof
• CPC Class Codes: H04N 23/54 Mounting of pick-up tubes, electronic image sensors, deviation or focusing coils
• CPC Class Codes: H04N 23/60 Control of cameras or camera modules
• CPC Class Codes: H04N 23/73 by influencing the exposure time
• CPC Class Codes: H04N 25/40 Extracting pixel data from image sensors by controlling scanning circuits, e.g. by modifying the number of pixels sampled or to be sampled
• CPC Class Codes: H04N 25/48 Increasing resolution by shifting the sensor relative to the scene
• CPC Class Codes: Y02E 10/50 Photovoltaic [PV] energy

Abstract

An imaging system includes an imaging optical system, an imaging device, an actuator, and control circuitry. The actuator changes a relative position of a plurality of pixel cells and an image of a subject. The pixel cells have variable sensitivity, and include a photoelectric converter and a charge accumulation region. The control circuitry sets the relative position to a first position, and also sets the sensitivity of each pixel cell to a first sensitivity. A first signal charge obtained at the photoelectric converter is accumulated in the charge accumulation region. The relative position is set to a second position different from the first position, and also the sensitivity of each pixel cell is set to a second sensitivity different from the first sensitivity. A second signal charge obtained at the photoelectric converter is accumulated in the charge accumulation region in addition to the first signal charge.

Claims

1. An imaging system, comprising:

an imaging optical system that images an image of a subject;

an imaging device including a plurality of pixel cells;

an actuator that changes a relative position of the plurality of pixel cells and the image of the subject; and

control circuitry that controls the imaging device and the actuator,

wherein the plurality of pixel cells each have variable sensitivity,

wherein the plurality of pixel cells each include

a photoelectric converter that converts light of the image of the subject into a signal charge, and

a charge accumulation region that accumulates the signal charge obtained at the photoelectric converter,

and wherein the control circuitry

sets the relative position to a first position, and also sets the sensitivity of each of the plurality of pixel cells to a first sensitivity, to cause a first signal charge obtained at the photoelectric converter to be accumulated in the charge accumulation region in each of the plurality of pixel cells, and

sets the relative position to a second position that is different from the first position, and also sets the sensitivity of each of the plurality of pixel cells to a second sensitivity that is different from the first sensitivity, to cause a second signal charge obtained at the photoelectric converter to be accumulated in the charge accumulation region in each of the plurality of pixel cells in addition to the first signal charge.

2. The imaging system according to claim 1,

wherein the control circuitry

synchronously performs the setting of the relative position to the first position, and the setting of the sensitivity of each of the plurality of pixel cells to the first sensitivity, and

synchronously performs the setting of the relative position to the second position, and the setting of the sensitivity of each of the plurality of pixel cells to the second sensitivity.

3. The imaging system according to claim 1,

wherein the control circuitry performs the setting to the second position by changing the relative position from the first position to the second position by an integer multiple of a center-to-center distance between two adjacent pixel cells out of the plurality of pixel cells.

8. The imaging system according to claim 1,

wherein the plurality of pixel cells are laid out two-dimensionally in a row direction and a column direction.

9. The imaging system according to claim 1,

wherein the control circuitry performs the setting of the sensitivity of each of the plurality of pixel cells all at once.

10. The imaging system according to claim 1,

wherein each of the plurality of pixel cells includes a first sub-pixel and a second sub-pixel,

wherein the first sub-pixel of each of the plurality of pixel cells includes the photoelectric converter and the charge accumulation region,

wherein the sensitivity of each of the plurality of pixel cells is the sensitivity of the first sub-pixel,

wherein the second sub-pixel of each of the plurality of pixel cells includes a second photoelectric converter and a second charge accumulation region,

and wherein the control circuitry sets or changes sensitivity of the second sub-pixel independently from the sensitivity of the first sub-pixel in each of the plurality of pixel cells.

11. The imaging system according to claim 1,

wherein the photoelectric converter of each of the plurality of pixel cells includes:

a pixel electrode connected to the charge accumulation region;

an opposing electrode that transmits light; and

a photoelectric conversion layer disposed between the pixel electrode and the opposing electrode.

12. The imaging system according to claim 11,

wherein the imaging device further includes

a voltage applying circuit that applies voltage across the pixel electrode and the opposing electrode included in the photoelectric converter of each of the plurality of pixel cells,

and wherein the control circuitry changes the sensitivity of each of the plurality of pixel cells by changing the voltage that the voltage applying circuit applies.

4. An imaging system, comprising:

an imaging optical system that images an image of a subject;

an imaging device including a plurality of pixel cells;

an actuator that changes a relative position of the plurality of pixel cells and the image of the subject; and

control circuitry that controls the imaging device and the actuator,

wherein the plurality of pixel cells each have variable sensitivity,

wherein the plurality of pixel cells each include

a photoelectric converter that converts light of the image of the subject into a signal charge, and

a charge accumulation region that accumulates the signal charge obtained at the photoelectric converter,

and wherein, during one exposure period, the control circuitry changes the relative position from a first position to a second position that is different from the first position, and changes the sensitivity of each of the plurality of pixel cells from a first sensitivity to a second sensitivity that is different from the first sensitivity, to cause a third signal charge obtained at the photoelectric converter to be accumulated at the charge accumulation region, in each of the plurality of pixel cells.

5. The imaging system according to claim 4,

wherein the control circuitry changes the relative position from the first position to the second position in a continuous manner.

6. The imaging system according to claim 4,

wherein the control circuitry changes the sensitivity of each of the plurality of pixel cells from the first sensitivity to the second sensitivity in a continuous manner.

7. The imaging system according to claim 4,

wherein the control circuitry performs the changing of the relative position and the changing of the sensitivity of each of the plurality of pixel cells in a synchronous manner.

13. An imaging method in an imaging system, the imaging system comprising:

an imaging optical system that images an image of a subject;

an imaging device including a plurality of pixel cells;

an actuator that changes a relative position of the plurality of pixel cells and the image of the subject; and

control circuitry that controls the imaging device and the actuator, wherein the plurality of pixel cells each have variable sensitivity, and the plurality of pixel cells each include;

a photoelectric converter that converts light of the image of the subject into a signal charge; and

a charge accumulation region that accumulates the signal charge obtained at the photoelectric converter,

the imaging method comprising;

setting the relative position to a first position;

setting the sensitivity of each of the plurality of pixel cells to a first sensitivity;

causing a first signal charge obtained at the photoelectric converter to be accumulated in the charge accumulation region in each of the plurality of pixel cells in a state where the relative position is set to the first position and the sensitivity of each of the plurality of pixel cells set to the first sensitivity;

setting the relative position to a second position that is different from the first position;

setting the sensitivity of each of the plurality of pixel cells to a second sensitivity that is different from the first sensitivity; and

causing a second signal charge obtained at the photoelectric converter to be accumulated in the charge accumulation region in each of the plurality of pixel cells in addition to the first signal charge, in a state where the relative position is set to the second position and the sensitivity of each of the plurality of pixel cells set to the second sensitivity.

14. The imaging method according to claim 13,

wherein the setting to the second position is performed by changing the relative position from the first position to the second position by an integer multiple of a center-to-center distance between two adjacent pixel cells out of the plurality of pixel cells.

15. The imaging method according to claim 13,

wherein the imaging system further comprises a mechanical shutter that shields imaging device from light,

and wherein the setting to the second position is performed by changing the relative position from the first position to the second position in a state where the imaging device is shielded from light by the mechanical shutter.

16. The imaging method according to claim 13,

wherein the setting to the second position is performed by changing the relative position from the first position to the second position in a state where the sensitivity of each of the plurality of pixel cells set to zero.

17. The imaging method according to claim 13, further comprising:

performing an N count (where N is an integer of 2 or greater) of settings of the relative position, including the setting to the first position and the setting to the second position, in which the relative position is set to an i'"'"'th (where i is an integer of 1 to N) position in an i'"'"'th setting of the relative position;

performing an N count of settings of the sensitivity, including the setting to the first sensitivity and the setting to the second sensitivity, in which the sensitivity of each of the plurality of pixel cells is set to an i'"'"'th sensitivity in an i'"'"'th setting of the sensitivity;

performing an N count of accumulations, including the accumulation of the first signal charge and the accumulation of the second signal charge, in which the i'"'"'th signal charge obtained at the photoelectric converter is accumulated at the charge accumulation region at each of the plurality of pixel cells in a state where the relative position is set to the i'"'"'th position and the sensitivity of each of the plurality of pixel cells is set to the i'"'"'th sensitivity at the i'"'"'th accumulation; and

obtaining a piece of image data using one or more pieces of image data obtained by the N count of the accumulations, the piece of image data being equivalent to a piece of image data obtainable by predetermined first convolution processing,

wherein the i'"'"'th position corresponds to a position of an i'"'"'th coefficient included in an N count of coefficients out of a plurality of coefficients in the first convolution processing,

and wherein the i'"'"'th sensitivity corresponds to the value of the i'"'"'th coefficient.

18. The imaging method according to claim 17,

wherein the one or more pieces of image data includes a first piece of image data and a second piece of image data,

wherein the N is an integer of 3 or greater,

wherein the N count of coefficients is made up of an M (wherein M is an integer of 1 or greater but smaller than N) count of coefficients having a positive value, and an (N−

M) count of coefficients having a negative value,

wherein the first piece of image data corresponds to a sum value of signal charges accumulated in the charge accumulation region of each of the plurality of pixel cells, by the M count of accumulations corresponding to the M count of coefficients,

wherein the second piece of image data corresponds to a sum value of signal charges accumulated in the charge accumulation region of each of the plurality of pixel cells, by an (N−

M) count of accumulations corresponding to the (N−

M) count of negative coefficients,

and wherein the piece of image data equivalent to the piece of image data obtainable by the first convolution processing is obtained by subtracting the second piece of image data from the first piece of image data.

19. The imaging method according to claim 13, further comprising:

performing an N count (where N is an integer of 4 or greater) of settings of the relative position, including the setting to the first position and the setting to the second position, in which the relative position is set to an i'"'"'th (where i is an integer of 1 to N) position in an i'"'"'th setting of the relative position;

performing an N count of settings of the sensitivity, including the setting to the first sensitivity and the setting to the second sensitivity, in which the sensitivity of each of the plurality of pixel cells is set to an i'"'"'th sensitivity in an i'"'"'th setting of the sensitivity;

performing an N count of accumulations, including the accumulation of the first signal charge and the accumulation of the second signal charge, in which the i'"'"'th signal charge obtained at the photoelectric converter is accumulated at the charge accumulation region at each of the plurality of pixel cells in a state where the relative position is set to the i'"'"'th position and the sensitivity of each of the plurality of pixel cells is set to the i'"'"'th sensitivity at the i'"'"'th accumulation;

performing a setting to a sensitivity corresponding to a sum value of an offset value and one of a plurality of coefficients of predetermined first convolution processing, in each of an M (wherein M is an integer of 2 or greater but smaller than N) counts of settings, out of the N counts of settings of the sensitivity;

performing a setting to sensitivity corresponding to the offset value in each of the (N−

M) count of settings, out of the N counts of settings of the sensitivity;

obtaining a first piece of image data by an M count of accumulations corresponding to the M count of settings of the sensitivity, the first piece of image data being equivalent to a piece of image data obtainable by convolution processing using a plurality of coefficients obtainable by adding an offset value to all coefficients of the first convolution processing;

obtaining a second piece of image data by an (N−

M) count of accumulations corresponding to the (N−

M) count of settings of the sensitivity, the second piece of image data being equivalent to a piece of image data obtainable by convolution processing using the offset value as all coefficients; and

obtaining a piece of image data by subtracting the second piece of image data from the first piece of image data, the piece of image data being equivalent to a piece of image data obtainable by the first convolution processing.

20. The imaging method according to claim 17,

wherein the value of the N count of coefficients used for the N count of settings of the sensitivity is not 0.

21. The imaging method according to claim 17, further comprising:

obtaining a piece of image data equivalent to a piece of image data obtainable by second convolution processing that is different from the first convolution processing,

wherein each of the plurality of pixel cells includes a first sub-pixel and a second sub-pixel,

wherein the first sub-pixel of each of the plurality of pixel cells includes the photoelectric converter and the charge accumulation region,

wherein the sensitivity of each of the plurality of pixel cells is sensitivity of the first sub-pixel,

wherein the second sub-pixel of each of the plurality of pixel cells includes a second photoelectric converter and a second charge accumulation region,

wherein, in each of the N count of settings of the sensitivity, sensitivity of the second sub-pixel in each of the plurality of pixel cells is further set,

wherein, in at least one of the N count of settings of the sensitivity, a setting value of the sensitivity of the first sub-pixel is different from a setting value of the sensitivity of the second sub-pixel,

wherein in each of the N count of accumulations, a signal charge obtained at the second photoelectric converter of the second sub-pixel is accumulated in the second charge accumulation region, in each of the plurality of pixel cells,

and wherein the piece of image data equivalent to the piece of image data obtainable by the first convolution processing is obtained using one or more pieces of image data obtained by the plurality of first sub-pixels by the N count of accumulations, and the piece of image data equivalent to the piece of image data obtainable by the second convolution processing is obtained using one or more other images obtained by the plurality of second sub-pixels by the N count of accumulations.

22. The imaging method according to claim 13, further comprising:

performing an N count (where N is an integer of 2 or greater) of settings of the relative position, including the setting to the first position and the setting to the second position, in which the relative position is set to an i'"'"'th (where i is an integer of 1 to N) position in an i'"'"'th setting of the relative position;

performing an N count of settings of the sensitivity, including the setting to the first sensitivity and the setting to the second sensitivity, in which the sensitivity of each of the plurality of pixel cells is set to an i'"'"'th sensitivity in an i'"'"'th setting of the sensitivity, and the exposure time is set to an i'"'"'th exposure time;

performing an N count of accumulations, including the accumulation of the first signal charge and the accumulation of the second signal charge, in which the i'"'"'th signal charge obtained at the photoelectric converter in the i'"'"'th exposure time is accumulated at the charge accumulation region at each of the plurality of pixel cells in a state where the relative position is set to the i'"'"'th position and the sensitivity of each of the plurality of pixel cells is set to the i'"'"'th sensitivity at the i'"'"'th accumulation; and

obtaining a piece of image data using one or more pieces of image data obtained by the N count of the accumulations, the piece of image data being equivalent to a piece of image data obtainable by predetermined first convolution processing.

23. An imaging method in an imaging system, the imaging system comprising:

an imaging optical system that images an image of a subject;

an imaging device including a plurality of pixel cells;

an actuator that changes a relative position of the plurality of pixel cells and the image of the subject; and

control circuitry that controls the imaging device and the actuator, wherein the plurality of pixel cells each have variable sensitivity, and the plurality of pixel cells each include;

a photoelectric converter that converts light of the image of the subject into a signal charge; and

a charge accumulation region that accumulates the signal charge obtained at the photoelectric converter,

the imaging method comprising;

performing, during one exposure period,

changing the relative position from a first position to a second position that is different from the first position;

changing the sensitivity of each of the plurality of pixel cells from a first sensitivity to a second sensitivity that is different from the first sensitivity; and

causing a third signal charge obtained at the photoelectric converter to be accumulated at the charge accumulation region, in each of the plurality of pixel cells.

24. The imaging method according to claim 23,

wherein the relative position is changed from the first position to the second position in a continuous manner.

25. The imaging system according to claim 23,

wherein the sensitivity of each of the plurality of pixel cells is changed from the first sensitivity to the second sensitivity in a continuous manner.