Device and method for image registration, and a nontransitory recording medium

• US 10,102,638 B2
• Filed: 02/02/2017
• Issued: 10/16/2018
• Est. Priority Date: 02/05/2016
• Status: Active Grant

• Current Assignee: Fujifilm Corporation (Fujifilm Holdings Corporation)
• Original Assignee: Fujifilm Corporation (Fujifilm Holdings Corporation)
• Inventors: Ito, Hirotaka
• Primary Examiner(s): Baker, Charlotte M
• Application Number:
  US15/423,022
• Publication Number:
  US 20170228877A1
• Time in Patent Office: 621 Days
• Field of Search: 382128, 382103, 3482221, 34833312, 3482118, 3405391, 34030916, 340540, 3405731, 222 28
• CPC Class Codes: A61B 2034/101 Computer-aided simulation of surgical operations
• CPC Class Codes: A61B 2090/365 augmented reality, i.e. correlating a live optical image with another image
• CPC Class Codes: A61B 2090/374 NMR or MRI
• CPC Class Codes: A61B 2090/3762 using computed tomography systems [CT]
• CPC Class Codes: A61B 34/10 Computer-aided planning, simulation or modelling of surgical operations
• CPC Class Codes: A61B 5/055 involving electronic [EMR] or nuclear [NMR] magnetic resonance, e.g. magnetic resonance imaging
• CPC Class Codes: A61B 6/032 Transmission computed tomography [CT]
• CPC Class Codes: A61B 6/037 Emission tomography
• CPC Class Codes: A61B 6/463 characterised by displaying multiple images or images and diagnostic data on one display
• CPC Class Codes: A61B 6/5223 generating planar views from image data, e.g. extracting a coronal view from a 3D image
• CPC Class Codes: A61B 6/5229 combining image data of a patient, e.g. combining a functional image with an anatomical image
• CPC Class Codes: G06T 2207/10028 Range image; Depth image; 3D point clouds
• CPC Class Codes: G06T 2207/10081 Computed x-ray tomography [CT]
• CPC Class Codes: G06T 2207/10088 Magnetic resonance imaging [MRI]
• CPC Class Codes: G06T 2207/10132 Ultrasound image
• CPC Class Codes: G06T 7/0012 Biomedical image inspection
• CPC Class Codes: G06T 7/30 Determination of transform parameters for the alignment of images, i.e. image registration
• CPC Class Codes: G06T 7/33 using feature-based methods

Abstract

A first registration unit performs first registration between a live view and an associated image associated with an object to be imaged. A second registration unit performs second registration between the object captured in the live view and the associated image based on the result of the first registration. At least while the second registration is performed, a display control unit superimposes the associated image on the object captured in the live view, and displays an enlarged view of a partial region of the live view, on which the associated image is superimposed. The second registration unit performs the second registration using a region of the live view larger than the partial region while the enlarged view of the partial region of the live view is displayed.

Claims

1. An image registration device comprising:

an image obtaining unit configured to obtain a live view comprising two or more images taken at different times, the live view capturing an object to be imaged;

an associated image obtaining unit configured to obtain an associated image associated with the object;

a display control unit configured to display the live view and the associated image on a display unit;

a first registration unit configured to perform, according to a first registration instruction, first registration between the object captured in the live view and the associated image; and

a second registration unit configured to perform second registration between the object captured in the live view and the associated image based on the result of the first registration,

wherein, at least while the second registration is performed, the display control unit superimposes the associated image on the object captured in the live view to achieve a registration between the associated image and the live view, and displays an enlarged view of a partial region of the live view, on which the associated image is superimposed, on the display unit, and

the second registration unit performs the second registration using a region of the live view larger than the partial region while the enlarged view of the partial region of the live view is displayed.

2. The image registration device as claimed in claim 1 further comprising an input unit configured to receive specification of the partial region.

3. The image registration device as claimed in claim 2, wherein the display control unit extracts the specified partial region from the live view, and enlarges only the extracted partial region to display an enlarged view of the extracted partial region on the display unit.

4. The image registration device as claimed in claim 1, wherein the second registration unit performs the second registration using the entire region of the live view while the enlarged view of the partial region of the live view is displayed.

5. The image registration device as claimed in claim 1, wherein, if the object includes at least one structure, the associated image is an image representing three-dimensional shapes of the object and the at least one structure.

6. An image registration method comprising the steps of:

obtaining a live view comprising two or more images taken at different times, the live view capturing an object to be imaged;

obtaining an associated image associated with the object;

displaying the live view and the associated image on a display unit;

performing, according to a first registration instruction, first registration between the object captured in the live view and the associated image;

performing second registration between the object captured in the live view and the associated image based on the result of the first registration;

superimposing the associated image on the object captured in the live view to achieve a registration between the associated image and the live view, and displaying an enlarged view of a partial region of the live view, on which the associated image is superimposed, on the display unit at least while the second registration is performed; and

performing the second registration using a region of the live view larger than the partial region while the enlarged view of the partial region of the live view is displayed.

7. A non-transitory recording medium with an image registration program recorded thereon, the image registration program causing a computer to execute the steps of:

obtaining a live view comprising two or more images taken at different times, the live view capturing an object to be imaged;

obtaining an associated image associated with the object;

displaying the live view and the associated image on a display unit;

performing, according to a first registration instruction, first registration between the object captured in the live view and the associated image;

performing second registration between the object captured in the live view and the associated image based on the result of the first registration;

superimposing the associated image on the object captured in the live view to achieve a registration between the associated image and the live view, and displaying an enlarged view of a partial region of the live view, on which the associated image is superimposed, on the display unit at least while the second registration is performed; and

performing the second registration using a region of the live view larger than the partial region while the enlarged view of the partial region of the live view is displayed.