Image acquisition device, image processing device, image processing method, image processing program, and storage medium for generating high resolution image from low resolution images

• US 10,477,236 B2
• Filed: 11/10/2017
• Issued: 11/12/2019
• Est. Priority Date: 05/21/2015
• Status: Active Grant

• Current Assignee: Olympus Corporation
• Original Assignee: Olympus Corporation
• Inventors: Furukawa, Eiji
• Primary Examiner(s): Allison, Andrae S
• Application Number:
  US15/809,947
• Publication Number:
  US 20180084274A1
• Time in Patent Office: 732 Days
• Field of Search: None
• CPC Class Codes: G06T 1/0007 Image acquisition
• CPC Class Codes: G06T 2207/20016 Hierarchical, coarse-to-fine, multiscale or multiresolution image processing; Pyramid transform
• CPC Class Codes: G06T 2207/20021 Dividing image into blocks, subimages or windows
• CPC Class Codes: G06T 2207/20224 Image subtraction
• CPC Class Codes: G06T 5/50 using two or more images, e.g. averaging or subtraction
• CPC Class Codes: G06T 7/11 Region-based segmentation
• CPC Class Codes: G06T 7/207 for motion estimation over a hierarchy of resolutions multi-resolution motion estimation or hierarchical motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/53
• CPC Class Codes: G06T 7/254 involving subtraction of images
• CPC Class Codes: H04N 19/43 Hardware specially adapted for motion estimation or compensation
• CPC Class Codes: H04N 19/53 Multi-resolution motion estimation; Hierarchical motion estimation

Abstract

An image acquisition device includes: an image acquisition unit that acquires images of a subject in time series; a hierarchical motion-detection unit that can detect a motion vector between the images acquired by the image acquisition unit, by using images at two or more levels having different resolutions; a pixel-value-change detecting unit that divides the images acquired by the image acquisition unit into a plurality of partial regions and that detects a temporal change of the pixel value at a detection pixel set in each of the partial regions; and a hierarchical-motion-detection setting unit that sets a larger number of levels used in the hierarchical motion-detection unit, for any of the partial regions that includes the detection pixel at which the temporal change detected by the pixel-value-change detecting unit is larger.

Claims

1. An image acquisition device comprising:

an image acquisition circuit that acquires images of a subject in time series;

a hierarchical motion-detection circuit that detects a motion vector between the images acquired by the image acquisition circuit, by using images at two or more levels having different resolutions;

a pixel-value-change detecting circuit that divides the images acquired by the image acquisition circuit into a plurality of partial regions and that detects a temporal change of a pixel value at a detection pixel set in each of the partial regions; and

a hierarchical-motion-detection setting circuit that increases a number of levels to be used in the hierarchical motion-detection circuit, for any of the partial regions that includes the detection pixel at which the temporal change detected by the pixel-value-change detecting circuit is larger than a predetermined threshold, in accordance with an increase in the temporal change.

2. The image acquisition device according to claim 1, wherein the image acquisition circuit and the pixel-value-change detecting circuit are provided in an image acquisition element.

3. The image acquisition device according to claim 1, wherein the pixel-value-change detecting circuit is provided with:

a memory circuit that stores a one-frame-previous image acquired by the image acquisition circuit; and

an arithmetic circuit that calculates, as the temporal change, a difference between the one-frame-previous image stored in the memory circuit and a current image.

4. The image acquisition device according to claim 1, wherein the hierarchical-motion-detection setting circuit prevents motion detection processing from being performed in the hierarchical motion-detection circuit for any of the partial regions that includes the detection pixel at which the temporal change detected by the pixel-value-change detecting circuit is equal to or less than the predetermined threshold.

5. The image acquisition device according to claim 1, wherein the hierarchical-motion-detection setting circuit expands a search range for the motion detection processing, which is performed in the hierarchical motion-detection circuit, as a magnitude of the temporal change detected by the pixel-value-change detecting circuit becomes larger.

6. An image processing device comprising:

a hierarchical motion-detection circuit that detects a motion vector between images acquired in time series, by using images at two or more levels having different resolutions;

a memory circuit that stores an image acquired one frame before a current image;

an arithmetic circuit that calculates a difference between the image stored in the memory circuit and the current image; and

a hierarchical motion-detection setting circuit that increases a number of levels used in the hierarchical motion-detection circuit, for a partial region that includes a detection pixel at which the difference calculated by the arithmetic circuit is larger than a predetermined threshold, in accordance with an increase in the difference.

7. The image processing device according to claim 6, wherein the hierarchical motion-detection setting circuit prevents motion detection processing from being performed in the hierarchical motion-detection circuit for a partial region which includes the detection pixel at which the difference calculated by the arithmetic circuit is equal to or less than the predetermined threshold.

8. The image processing device according to claim 6, wherein the hierarchical motion-detection setting circuit expands a search range for motion detection processing performed in the hierarchical motion-detection circuit as a magnitude of the difference calculated by the arithmetic circuit becomes larger.

9. An image processing method comprising:

calculating a difference between two images acquired in time series;

increasing a number of levels set for a partial region that includes a detection pixel at which the calculated difference is larger than a predetermined threshold, in accordance with an increase in the difference; and

detecting, for each partial region, a motion vector between the two images by using the set number of levels.

10. The image processing method according to claim 9, wherein, in the detecting, the motion vector is detected after excluding a partial region which includes the detection pixel at which the calculated difference is equal to or less than the predetermined threshold.

11. The image processing method according to claim 9, wherein, in the detecting, the motion vector is detected after expanding a search range as the calculated difference becomes larger.

12. A non-transitory computer-readable storage medium having recorded thereon an image processing program for causing a computer to execute processes comprising:

calculating a difference between two images acquired in time series;

increasing a number of levels set for a partial region that includes a detection pixel at which the calculated difference is larger, than a predetermined threshold, in accordance with an increase in the difference; and

detecting, for each partial region, a motion vector between the two images by using the set number of levels.