APPARATUS, METHOD AND RECORDING MEDIUM FOR IMAGE PROCESSING

• US 20130083171A1
• Filed: 10/04/2012
• Published: 04/04/2013
• Est. Priority Date: 10/04/2011
• Status: Active Grant

• Current Assignee: Morpho Incorporated
• Original Assignee: Morpho Incorporated
• Inventors: HABUKA, Kensuke, ABE, Go
• Granted Patent: US 9,117,271 B2
• Time in Patent Office: Days
• US Class Current: 348/47
• CPC Class Codes: G06T 2207/10012 Stereo images
• CPC Class Codes: G06T 2207/10021 Stereoscopic video; Stereoscopic image sequence
• CPC Class Codes: G06T 2207/20021 Dividing image into blocks, subimages or windows
• CPC Class Codes: G06T 2207/20201 Motion blur correction
• CPC Class Codes: G06T 5/50 using two or more images, e.g. averaging or subtraction
• CPC Class Codes: G06T 5/73 Deblurring; Sharpening
• CPC Class Codes: G06T 7/248 involving reference images or patches
• CPC Class Codes: H04N 23/6811 based on the image signal
• CPC Class Codes: H04N 23/6842 by controlling the scanning position, e.g. windowing

Abstract

An image processing apparatus 1 includes: a placement information acquisition section 40 which acquires, for each image sensor, placement information of the image sensors with respect to the imaging section 10; a movement information acquisition section 30 which acquires movement information of the imaging device; and an output area acquisition section which acquires, for every image sensor, position information of a second output area within the second frame image which corresponds to a first output area within the first frame image. The output area acquisition section acquires: a second output area A12 of a base image sensor based on position information of a first output area A11 determined by the placement information and based on the movement information; and a second output area A22 of another image sensor based on the placement information of the base image sensor, the placement information of the other image sensor, and the movement information.

Claims

1. An image processing apparatus which performs image processing of a plurality of frame images generated by an imaging device having a plurality of image sensors each continuously generating a first frame image and a second frame image in sequence, comprising:

a placement information acquisition section which acquires, for each image sensor, placement information indicating placement conditions of the image sensors with respect to the imaging device;

a movement information acquisition section which acquires movement information of the imaging device between the first frame image and the second frame image; and

an output area acquisition section which acquires, for each image sensor, position information of a second output area within the second frame image, the second output area corresponding to a first output area within the first frame image, wherein

the output area acquisition section acquires;

as a reference base, the second output area of a base image sensor among the plurality of image sensors based on the position information of the first output area determined by the placement information and based on the movement information; and

the second output area of an image sensor being different from the base image sensor, among the plurality of image sensors based on the placement information of the base image sensor, the placement information of another image sensor, and the movement information.

2. The image processing apparatus according to claim 1, comprising:

a determination section which determines for all the image sensors whether or not the position information of the second output area is positioned within the second frame image, wherein

when the position information of the second output area is not positioned within the second frame image, the output area acquisition section sets moving amount obtained by a difference between the position information of the first output area and the position information of the second output area to be smaller until the position information of the second output area is positioned within the second frame image in all the image sensors.

3. The image processing apparatus according to claim 1, wherein

the movement information acquisition section acquires the movement information from the first frame image and the second frame image generated in the base image sensor.

4. The image processing apparatus according to claim 1, wherein

the movement information acquisition section acquires the movement information by a gyro sensor.

5. The image processing apparatus according to claim 1, wherein

the movement information acquisition section acquires as the movement information a weighted average of the movement information acquired from the first frame image and the second frame image generated by the plurality of image sensors.

6. The image processing apparatus according to claim 1, wherein

the base image sensor is selected out of the plurality of image sensors based on image pickup conditions.

7. The image processing apparatus according to claim 1, wherein

the output area acquisition section uses an area other than the output area being needed because of using the plurality of image sensors, as an area for use in correction with movement.

10. The image processing apparatus according to claim 2, wherein

the movement information acquisition section acquires the movement information from the first frame image and the second frame image generated in the base image sensor.

11. The image processing apparatus according to claim 2, wherein

the movement information acquisition section acquires the movement information by a gyro sensor.

12. The image processing apparatus according to claim 2, wherein

the movement information acquisition section acquires as the movement information a weighted average of the movement information acquired from the first frame image and the second frame image generated by the plurality of image sensors.

13. The image processing apparatus according to claim 2, wherein

the base image sensor is selected out of the plurality of image sensors based on image pickup conditions.

14. The image processing apparatus according to claim 3, wherein

the base image sensor is selected out of the plurality of image sensors based on image pickup conditions.

15. The image processing apparatus according to claim 4, wherein

the base image sensor is selected out of the plurality of image sensors based on image pickup condition

16. The image processing apparatus according to claim 5, wherein

the base image sensor is selected out of the plurality of image sensors based on image pickup condition

17. The image processing apparatus according to claim 2, wherein

the output area acquisition section uses an area other than the output area which is needed for provision of the plurality of image sensors as an area for use in correction with movement.

18. The image processing apparatus according claim 3, wherein

the output area acquisition section uses an area other than the output area which is needed for provision of the plurality of image sensors as an area for use in correction with movement.

19. The image processing apparatus according to claim 4, wherein

the output area acquisition section uses an area other than the output area which is needed for provision of the plurality of image sensors as an area for use in correction with movement.

20. The image processing apparatus according claim 5, wherein

the output area acquisition section uses an area other than the output area which is needed for provision of the plurality of image sensors as an area for use in correction with movement.

8. An image processing method for performing image processing of a plurality of frame images generated by an imaging device having a plurality of image sensors which continuously generate a first frame image and a second frame image in sequence, comprising:

a placement information acquisition step of acquiring, for every image sensor, placement information indicating placement conditions of the image sensors with respect to the imaging device;

a movement information acquisition step of acquiring movement information of the imaging device between the first frame image and the second frame image; and

an output area acquisition step of acquiring, for every image sensor, position information of a second output area within the second frame image which corresponds to a first output area within the first frame image, wherein

in the output area acquisition step,

as a reference base, the second output area of a base image sensor among the plurality of image sensors is acquired based on position information of the first output area determined by the placement information and based on the movement information, and

the second output area of an image sensor being different from the base image sensor, among the plurality of image sensors is acquired based on placement information of the base image sensor, placement information of the other image sensor, and the movement information.

9. A storage medium having stored thereon an image processing program for operating a computer to perform image processing of a plurality of frame images generated by an imaging device having a plurality of image sensors which continuously generate a first frame image and a second frame image in sequence, the program causing the computer to function as:

a placement information acquisition section which acquires, for every image sensor, placement information indicating placement conditions of the image sensors with respect to the imaging device;

a movement information acquisition section which acquires movement information of the imaging device between the first frame image and the second frame image; and

an output area acquisition section which acquires, for every image sensor, position information of a second output area within the second frame image which corresponds to a first output area within the first frame image, wherein

the output area acquisition section acquires;

as a reference base, the second output area of a base image sensor among the plurality of image sensors based on position information of the first output area determined by the placement information and based on the movement information; and

the second output area of an image sensor being different from the base image sensor, among the plurality of image sensors based on placement information of the base image sensor, placement information of the other image sensor, and the movement information.