Method and apparatus for image registration using large deformation diffeomorphisms on a sphere

0Associated
Cases 
0Associated
Defendants 
0Accused
Products 
104Forward
Citations 
0
Petitions 
1
Assignment
First Claim
1. A method for registering a template image with a target image, the template image containing a plurality of points and the target image containing a plurality of points, comprising:
 defining at least one manifold landmark point in said template image;
identifying at least one point in said target image corresponding to said defined at least one manifold landmark point;
selecting a coordinate frame suitable for spherical geometries;
computing a large deformation transform using said selected coordinate frame, at least one manifold landmark transformation operator, and at least one manifold landmark transformation boundary value, said large deformation transform relating said at least one manifold landmark point in said template image to said corresponding at least one point in said target image; and
registering said template image with said target image using said large deformation transform in the selected coordinate frame.
1 Assignment
0 Petitions
Accused Products
Abstract
An apparatus and method for image registration of a template image with a target image with large deformation. The apparatus and method involve computing a large deformation transform based on landmark manifolds, image data or both. The apparatus and method are capable of registering images with a small number of landmark points. Registering the images is accomplished by applying the large deformation transform.
215 Citations
Method of navigation in threedimensional image data  
Patent #
US 7,889,894 B2
Filed 11/22/2004

Current Assignee
Siemens Healthcare GMBH

Sponsoring Entity
Siemens AG

AUTOMATED SURFACEBASED ANATOMICAL ANALYSIS BASED ON ATLASBASED SEGMENTATION OF MEDICAL IMAGING  
Patent #
US 20110092794A1
Filed 07/07/2009

Current Assignee
Johns Hopkins University

Sponsoring Entity
Johns Hopkins University

Method and apparatus for modulating a radiation beam  
Patent #
US 7,957,507 B2
Filed 02/24/2006

Current Assignee
Accuray Incorporated

Sponsoring Entity
Patrick F. Cadman

Positioning measurement apparatus and method  
Patent #
US 7,961,931 B2
Filed 10/24/2007

Current Assignee
ENSKY TECHNOLOGY CO. LTD., ENSKY TECHNOLOGY SHENZHEN CO. LTD.

Sponsoring Entity
ENSKY TECHNOLOGY CO. LTD., ENSKY TECHNOLOGY SHENZHEN CO. LTD.

Mutual symmetry detection  
Patent #
US 7,936,928 B2
Filed 09/06/2007

Current Assignee
National Instruments Corporation

Sponsoring Entity
National Instruments Corporation

Estimating T2diffusion probability density functions from nuclear magnetic resonance diffusion modulated amplitude measurements  
Patent #
US 8,044,662 B2
Filed 08/14/2009

Current Assignee
Halliburton Energy Services Incorporated

Sponsoring Entity
Halliburton Energy Services Incorporated

ESTIMATING T2DIFFUSION PROBABILITY DENSITY FUNCTIONS FROM NUCLEAR MAGNETIC RESONANCE DIFFUSION MODULATED AMPLITUDE MEASUREMENTS  
Patent #
US 20100259258A1
Filed 08/14/2009

Current Assignee
Halliburton Energy Services Incorporated

Sponsoring Entity
Halliburton Energy Services Incorporated

Method for threedimensional modeling of the skull and internal structures thereof  
Patent #
US 7,720,519 B2
Filed 10/16/2003

Current Assignee
Elekta Neuromag Oy

Sponsoring Entity
ELEKTA OY

CONSISTENCY METRIC BASED IMAGE REGISTRATION  
Patent #
US 20100260392A1
Filed 12/12/2008

Current Assignee
Koninklijke Philips N.V.

Sponsoring Entity
Koninklijke Philips N.V.

System and method for automatically registering three dimensional cardiac images with electroanatomical cardiac mapping data  
Patent #
US 7,715,604 B2
Filed 01/12/2006

Current Assignee
Siemens Healthcare GMBH

Sponsoring Entity
Siemens Medical Solutions USA Incorporated

System and method of evaluating dose delivered by a radiation therapy system  
Patent #
US 7,839,972 B2
Filed 07/21/2006

Current Assignee
TOMOTHERAPY INCORPORATED

Sponsoring Entity
TOMOTHERAPY INCORPORATED

Method and system for evaluating delivered dose  
Patent #
US 7,643,661 B2
Filed 07/21/2006

Current Assignee
TOMOTHERAPY INCORPORATED

Sponsoring Entity
TOMOTHERAPY INCORPORATED

Method and system for evaluating quality assurance criteria in delivery of a treatment plan  
Patent #
US 7,773,788 B2
Filed 07/21/2006

Current Assignee
TOMOTHERAPY INCORPORATED

Sponsoring Entity
TOMOTHERAPY INCORPORATED

AUTOMATED IMAGE ANALYSIS FOR MAGNETIC RESONANCE IMAGING  
Patent #
US 20100284595A1
Filed 02/02/2009

Current Assignee
Johns Hopkins University

Sponsoring Entity
Johns Hopkins University

System and method for GPUbased 3D nonrigid registration  
Patent #
US 7,483,034 B2
Filed 02/22/2005

Current Assignee
Siemens Medical Solutions USA Incorporated

Sponsoring Entity
Siemens Medical Solutions USA Incorporated

Image guided surgery with dynamic image reconstruction  
Patent #
US 20090136099A1
Filed 11/26/2007

Current Assignee
Gearbox LLC

Sponsoring Entity
Gearbox LLC

System and method for tracking anatomical structures in three dimensional images  
Patent #
US 7,555,151 B2
Filed 09/01/2005

Current Assignee
Siemens Healthcare GMBH

Sponsoring Entity
Siemens Medical Solutions USA Incorporated

Method of placing constraints on a deformation map and system for implementing same  
Patent #
US 7,567,694 B2
Filed 07/21/2006

Current Assignee
TOMOTHERAPY INCORPORATED

Sponsoring Entity
TOMOTHERAPY INCORPORATED

Method and system for adapting a radiation therapy treatment plan based on a biological model  
Patent #
US 7,574,251 B2
Filed 07/21/2006

Current Assignee
TOMOTHERAPY INCORPORATED

Sponsoring Entity
TOMOTHERAPY INCORPORATED

Constructing a statistical shape model from twodimensional or threedimensional data  
Patent #
US 7,584,080 B2
Filed 12/06/2001

Current Assignee
Imorphics Limited

Sponsoring Entity
Imorphics Limited

System and method of generating contour structures using a dose volume histogram  
Patent #
US 7,609,809 B2
Filed 07/21/2006

Current Assignee
TOMOTHERAPY INCORPORATED

Sponsoring Entity
TOMOTHERAPY INCORPORATED

POINT SUBSELECTION FOR FAST DEFORMABLE POINTBASED IMAGING  
Patent #
US 20090279739A1
Filed 05/31/2006

Current Assignee
Koninklijke Philips N.V.

Sponsoring Entity
Koninklijke Philips N.V.

Method and system for processing data relating to a radiation therapy treatment plan  
Patent #
US 7,639,854 B2
Filed 07/21/2006

Current Assignee
TOMOTHERAPY INCORPORATED

Sponsoring Entity
TOMOTHERAPY INCORPORATED

Method of and system for predicting dose delivery  
Patent #
US 7,639,853 B2
Filed 07/21/2006

Current Assignee
TOMOTHERAPY INCORPORATED

Sponsoring Entity
TOMOTHERAPY INCORPORATED

POSITIONING MEASUREMENT APPARATUS AND METHOD  
Patent #
US 20080103726A1
Filed 10/24/2007

Current Assignee
ENSKY TECHNOLOGY CO. LTD., ENSKY TECHNOLOGY SHENZHEN CO. LTD.

Sponsoring Entity
ENSKY TECHNOLOGY CO. LTD., ENSKY TECHNOLOGY SHENZHEN CO. LTD.

METHOD AND APPARATUS FOR GUIDING TOWARDS TARGETS DURING MOTION  
Patent #
US 20080186378A1
Filed 11/13/2007

Current Assignee
International Game Technology

Sponsoring Entity
International Game Technology

Method and apparatus for 4dimensional image warping  
Patent #
US 7,430,323 B2
Filed 08/05/2004

Current Assignee
Trustees Of The University Of Pennsylvania

Sponsoring Entity
Trustees Of The University Of Pennsylvania

Process and System for Analysing Deformations in Motor Vehicles  
Patent #
US 20080267487A1
Filed 03/25/2005

Current Assignee
Fausto Siri

Sponsoring Entity
Fausto Siri

METHOD AND SYSTEM FOR EVALUATING QUALITY ASSURANCE CRITERIA IN DELIVERY OF A TREATMENT PLAN  
Patent #
US 20070041499A1
Filed 07/21/2006

Current Assignee
TOMOTHERAPY INCORPORATED

Sponsoring Entity
TOMOTHERAPY INCORPORATED

Image matching apparatus, method of matching images, and computer program product  
Patent #
US 20070041658A1
Filed 03/14/2006

Current Assignee
Toshiba Corporation

Sponsoring Entity
Toshiba Corporation

METHOD OF AND SYSTEM FOR PREDICTING DOSE DELIVERY  
Patent #
US 20070041495A1
Filed 07/21/2006

Current Assignee
TOMOTHERAPY INCORPORATED

Sponsoring Entity
TOMOTHERAPY INCORPORATED

SYSTEM AND METHOD OF RECOMMENDING A LOCATION FOR RADIATION THERAPY TREATMENT  
Patent #
US 20070104316A1
Filed 07/21/2006

Current Assignee
TOMOTHERAPY INCORPORATED

Sponsoring Entity
TOMOTHERAPY INCORPORATED

Method of navigation in threedimensional image data  
Patent #
US 20070115204A1
Filed 11/22/2004

Current Assignee
Siemens Healthcare GMBH

Sponsoring Entity
Siemens Healthcare GMBH

Rotational symmetry detection for configurations of discrete curves  
Patent #
US 7,212,673 B2
Filed 06/05/2003

Current Assignee
National Instruments Corporation

Sponsoring Entity
National Instruments Corporation

Elastic image registration  
Patent #
US 20070179377A1
Filed 12/10/2004

Current Assignee
Koninklijke Philips N.V.

Sponsoring Entity
Koninklijke Philips N.V.

Discrete curve symmetry detection  
Patent #
US 7,269,286 B2
Filed 06/05/2003

Current Assignee
National Instruments Corporation

Sponsoring Entity
National Instruments Corporation

Mutual Symmetry Detection  
Patent #
US 20070297681A1
Filed 09/06/2007

Current Assignee
National Instruments Corporation

Sponsoring Entity
National Instruments Corporation

Image registration method  
Patent #
US 20060008179A1
Filed 11/18/2003

Current Assignee
Universitat Zu Lubeck

Sponsoring Entity
Universitat Zu Lubeck

System and method for tracking anatomical structures in three dimensional images  
Patent #
US 20060064007A1
Filed 09/01/2005

Current Assignee
Siemens Healthcare GMBH

Sponsoring Entity
Siemens Healthcare GMBH

Method for threedimensional modeling of the skull and internal structures thereof  
Patent #
US 20060052687A1
Filed 10/16/2003

Current Assignee
Elekta Neuromag Oy

Sponsoring Entity
Elekta Neuromag Oy

Registration of first and second image data of an object  
Patent #
US 20060072808A1
Filed 09/16/2005

Current Assignee
MEDCOM GESELLSCHAFT FUR MEDIZINISCHE BILDVERARBEITUNG MBH, ESAOTE RUFFINO S.P.A.

Sponsoring Entity
MEDCOM GESELLSCHAFT FUR MEDIZINISCHE BILDVERARBEITUNG MBH, ESAOTE RUFFINO S.P.A.

Method and device for the registration of images  
Patent #
US 7,062,078 B2
Filed 11/02/2001

Current Assignee
Koninklijke Philips N.V.

Sponsoring Entity
Koninklijke Philips N.V.

Image processing apparatus for performing image restoration  
Patent #
US 7,079,179 B2
Filed 03/21/2002

Current Assignee
Minolta Corporation Limited

Sponsoring Entity
Minolta Corporation Limited

System and method for automatically registering three dimensional cardiac images with electroanatomical cardiac mapping data  
Patent #
US 20060159323A1
Filed 01/12/2006

Current Assignee
Siemens Healthcare GMBH

Sponsoring Entity
Siemens Healthcare GMBH

Digital radiograph analyzer and methods  
Patent #
US 7,113,162 B1
Filed 05/12/2000

Current Assignee
Bradley D. Beasley

Sponsoring Entity
Bradley D. Beasley

Image pattern matching utilizing discrete curve matching with a mapping operator  
Patent #
US 7,139,432 B2
Filed 10/03/2002

Current Assignee
National Instruments Corporation

Sponsoring Entity
National Instruments Corporation

Method and apparatus for 4dimensional image warping  
Patent #
US 20050031210A1
Filed 08/05/2004

Current Assignee
Trustees Of The University Of Pennsylvania

Sponsoring Entity
Trustees Of The University Of Pennsylvania

Statistical model  
Patent #
US 20050027492A1
Filed 12/06/2001

Current Assignee
Imorphics Limited

Sponsoring Entity
Imorphics Limited

System and method for GPUbased 3D nonrigid registration  
Patent #
US 20050190189A1
Filed 02/22/2005

Current Assignee
Siemens Medical Solutions USA Incorporated

Sponsoring Entity
Siemens Medical Solutions USA Incorporated

Realtime stabilization  
Patent #
US 20050285947A1
Filed 06/21/2004

Current Assignee
Intergraph Software Technologies Company

Sponsoring Entity
Intergraph Software Technologies Company

Method and system for assessment of biomarkers by measurement of response to stimulus  
Patent #
US 20040066955A1
Filed 10/02/2002

Current Assignee
VirtualScopics Inc.

Sponsoring Entity
VirtualScopics Inc.

Initializing modelbased interpretations of digital radiographs  
Patent #
US 20040101186A1
Filed 11/27/2002

Current Assignee
Canon Kabushiki Kaisha

Sponsoring Entity
Canon Kabushiki Kaisha

System and method for image pattern matching using a unified signal transform  
Patent #
US 6,807,305 B2
Filed 04/10/2001

Current Assignee
National Instruments Corporation

Sponsoring Entity
National Instruments Corporation

Method and system for assessment of biomarkers by measurement of response to stimulus  
Patent #
US 6,836,557 B2
Filed 10/02/2002

Current Assignee
VirtualScopics Inc.

Sponsoring Entity
VirtualScopicsLLC

Rotational symmetry detection for configurations of discrete curves  
Patent #
US 20040247182A1
Filed 06/05/2003

Current Assignee
National Instruments Corporation

Sponsoring Entity
National Instruments Corporation

Discrete curve symmetry detection  
Patent #
US 20040247181A1
Filed 06/05/2003

Current Assignee
National Instruments Corporation

Sponsoring Entity
National Instruments Corporation

System and method for image pattern matching using a unified signal transform  
Patent #
US 20030007690A1
Filed 04/10/2001

Current Assignee
National Instruments Corporation

Sponsoring Entity
National Instruments Corporation

Image pattern matching utilizing discrete curve matching with a mapping operator  
Patent #
US 20030198389A1
Filed 10/03/2002

Current Assignee
National Instruments Corporation

Sponsoring Entity
National Instruments Corporation

Method and device for localizing a structure in a measured data set  
Patent #
US 20030210820A1
Filed 05/07/2003

Current Assignee
Brainlab AG

Sponsoring Entity
Brainlab AG

Method and device for the registration of images  
Patent #
US 20020122576A1
Filed 11/02/2001

Current Assignee
Koninklijke Philips N.V.

Sponsoring Entity
Koninklijke Philips N.V.

Image processing apparatus  
Patent #
US 20020164082A1
Filed 03/21/2002

Current Assignee
Minolta Corporation Limited

Sponsoring Entity
Minolta Corporation Limited

Point subselection for fast deformable pointbased imaging  
Patent #
US 8,094,895 B2
Filed 05/31/2006

Current Assignee
Koninklijke Philips N.V.

Sponsoring Entity
Koninklijke Philips N.V.

3D ULTRASOUND SYSTEM FOR EXTENDING VIEW OF IMAGE AND METHOD FOR OPERATING THE 3D ULTRASOUND SYSTEM  
Patent #
US 20120065513A1
Filed 09/12/2011

Current Assignee
Samsung Medison Company Limited

Sponsoring Entity
Samsung Medison Company Limited

System and method of detecting a breathing phase of a patient receiving radiation therapy  
Patent #
US 8,229,068 B2
Filed 07/21/2006

Current Assignee
TOMOTHERAPY INCORPORATED

Sponsoring Entity
TOMOTHERAPY INCORPORATED

System and method of treating a patient with radiation therapy  
Patent #
US 8,232,535 B2
Filed 05/10/2006

Current Assignee
TOMOTHERAPY INCORPORATED

Sponsoring Entity
TOMOTHERAPY INCORPORATED

Elastic image registration  
Patent #
US 8,326,086 B2
Filed 12/10/2004

Current Assignee
Koninklijke Philips N.V.

Sponsoring Entity
Koninklijke Philips N.V.

Method and system for evaluating quality assurance criteria in delivery of a treatment plan  
Patent #
US 8,442,287 B2
Filed 08/10/2010

Current Assignee
TOMOTHERAPY INCORPORATED

Sponsoring Entity
TOMOTHERAPY INCORPORATED

Process and system for analyzing deformations in motor vehicles  
Patent #
US 8,515,152 B2
Filed 03/25/2005

Current Assignee
Fausto Siri

Sponsoring Entity
Fausto Siri

Method and apparatus for registration of an atlas to an image  
Patent #
US 8,687,917 B2
Filed 05/02/2006

Current Assignee
Agency for Science Technology and Research

Sponsoring Entity
Agency for Science Technology and Research

Automated image analysis for magnetic resonance imaging  
Patent #
US 8,731,256 B2
Filed 02/02/2009

Current Assignee
Johns Hopkins University

Sponsoring Entity
Johns Hopkins University

System and method of delivering radiation therapy to a moving region of interest  
Patent #
US 8,767,917 B2
Filed 07/21/2006

Current Assignee
TOMOTHERAPY INCORPORATED

Sponsoring Entity
TOMOTHERAPY INCORPORATED

System and method for radiation therapy treatment planning using a memetic optimization algorithm  
Patent #
US 8,792,614 B2
Filed 03/31/2010

Current Assignee
Matthew R. Witten, Owen Clancey

Sponsoring Entity
Matthew R. Witten, Owen Clancey

Methods and systems for assessing image quality in modeling of patient anatomic or blood flow characteristics  
Patent #
US 8,824,752 B1
Filed 01/24/2014

Current Assignee
Heartflow Inc.

Sponsoring Entity
Heartflow Inc.

Methods and systems for assessing image quality in modeling of patient anatomic or blood flow characteristics  
Patent #
US 8,831,314 B1
Filed 02/04/2014

Current Assignee
Heartflow Inc.

Sponsoring Entity
Heartflow Inc.

Methods and systems for assessing image quality in modeling of patient anatomic or blood flow characteristics  
Patent #
US 8,831,315 B1
Filed 02/04/2014

Current Assignee
Heartflow Inc.

Sponsoring Entity
Heartflow Inc.

Atlasbased analysis for imagebased anatomic and functional data of organism  
Patent #
US 8,838,201 B2
Filed 06/22/2011

Current Assignee
Johns Hopkins University

Sponsoring Entity
Johns Hopkins University

Methods and systems for assessing image quality in modeling of patient anatomic or blood flow characteristics  
Patent #
US 8,861,820 B2
Filed 02/04/2014

Current Assignee
Heartflow Inc.

Sponsoring Entity
Heartflow Inc.

Methods and systems for assessing image quality in modeling of patient anatomic or blood flow characteristics  
Patent #
US 9,008,405 B2
Filed 09/11/2014

Current Assignee
Heartflow Inc.

Sponsoring Entity
Heartflow Inc.

Image guided surgery with dynamic image reconstruction  
Patent #
US 9,076,203 B2
Filed 11/26/2007

Current Assignee
Gearbox LLC

Sponsoring Entity
The Invention Science Fund I L.L.C.

BIOIMAGING GRID  
Patent #
US 20150302171A1
Filed 12/31/2012

Current Assignee
Johns Hopkins University

Sponsoring Entity
Johns Hopkins University

System and method for evaluating lead stability of an implantable medical device  
Patent #
US 9,302,099 B2
Filed 05/04/2015

Current Assignee
Pacesetter Incorporated

Sponsoring Entity
Pacesetter Incorporated

Method and system to assess mechanical dyssynchrony based on motion data collected by a navigation system  
Patent #
US 9,301,713 B2
Filed 07/10/2014

Current Assignee
Pacesetter Incorporated

Sponsoring Entity
Pacesetter Incorporated

Multiple shell construction to emulate chamber contraction with a mapping system  
Patent #
US 9,307,931 B2
Filed 12/31/2008

Current Assignee
St Jude Medical Atrial Fibrillation Divison Incorporated

Sponsoring Entity
St Jude Medical Atrial Fibrillation Divison Incorporated

Method and system to measure cardiac motion using a cardiovascular navigation system  
Patent #
US 9,314,191 B2
Filed 07/10/2014

Current Assignee
Pacesetter Incorporated

Sponsoring Entity
Pacesetter Incorporated

Method and system to characterize motion data based on neighboring map points  
Patent #
US 9,364,170 B2
Filed 05/05/2014

Current Assignee
Pacesetter Incorporated

Sponsoring Entity
Pacesetter Incorporated

Method and system for displaying a three dimensional visualization of cardiac motion  
Patent #
US 9,380,940 B2
Filed 05/05/2014

Current Assignee
Pacesetter Incorporated

Sponsoring Entity
Pacesetter Incorporated

Electromagnetically actuated multileaf collimator  
Patent #
US 9,443,633 B2
Filed 02/19/2014

Current Assignee
Accuray Incorporated

Sponsoring Entity
TOMOTHERAPY INCORPORATED

Image processing device, system, image processing method, and image processing program  
Patent #
US 9,607,244 B2
Filed 02/19/2015

Current Assignee
Omron Corporation

Sponsoring Entity
Omron Corporation

Bioimaging grid  
Patent #
US 9,646,138 B2
Filed 12/31/2012

Current Assignee
Johns Hopkins University

Sponsoring Entity
Johns Hopkins University

Methods and systems for assessing image quality in modeling of patient anatomic or blood flow characteristics  
Patent #
US 9,672,615 B2
Filed 11/26/2014

Current Assignee
Heartflow Inc.

Sponsoring Entity
Heartflow Inc.

Method and system to equalizing cardiac cycle length between map points  
Patent #
US 9,700,233 B2
Filed 05/04/2015

Current Assignee
Pacesetter Incorporated

Sponsoring Entity
Pacesetter Incorporated

Automated surfacebased anatomical analysis based on atlasbased segmentation of medical imaging  
Patent #
US 9,730,615 B2
Filed 07/07/2009

Current Assignee
Johns Hopkins University

Sponsoring Entity
Johns Hopkins University

Radiation therapy imaging and delivery utilizing coordinated motion of gantry and couch  
Patent #
US 9,731,148 B2
Filed 07/21/2006

Current Assignee
TOMOTHERAPY INCORPORATED

Sponsoring Entity
TOMOTHERAPY INCORPORATED

Method and system to subdivide a mapping area for mechanical activation analysis  
Patent #
US 9,763,591 B2
Filed 05/04/2015

Current Assignee
Pacesetter Incorporated

Sponsoring Entity
Pacesetter Incorporated

Method and system to identify motion data associated with consistent electrical and mechanical behavior for a region of interest  
Patent #
US 9,814,406 B2
Filed 09/05/2014

Current Assignee
Pacesetter Incorporated

Sponsoring Entity
Pacesetter Incorporated

Three dimensional image scan for vehicle  
Patent #
US 9,824,453 B1
Filed 10/14/2015

Current Assignee
Allstate Insurance Company

Sponsoring Entity
Allstate Insurance Company

Methods and systems for assessing image quality in modeling of patient anatomic or blood flow characteristics  
Patent #
US 9,836,840 B2
Filed 06/17/2016

Current Assignee
Heartflow Inc.

Sponsoring Entity
Heartflow Inc.

Cardiac resynchronization system and method  
Patent #
US 9,861,823 B2
Filed 05/04/2015

Current Assignee
Pacesetter Incorporated

Sponsoring Entity
Pacesetter Incorporated

Method and system to determine cardiac cycle length in connection with cardiac mapping  
Patent #
US 9,895,076 B2
Filed 05/04/2015

Current Assignee
Pacesetter Incorporated

Sponsoring Entity
Pacesetter Incorporated

Patient positioning system  
Patent #
US 9,950,194 B2
Filed 09/09/2014

Current Assignee
Mevion Medical Systems Incorporated

Sponsoring Entity
Mevion Medical Systems Incorporated

Method and system for calculating strain from characterization data of a cardiac chamber  
Patent #
US 10,105,077 B2
Filed 05/05/2014

Current Assignee
Pacesetter Incorporated

Sponsoring Entity
Pacesetter Incorporated

System and method for rendering a motion model of a beating heart  
Patent #
US 10,271,794 B2
Filed 04/29/2015

Current Assignee
St. Jude Medical Cardiology Division Incorporated

Sponsoring Entity
St. Jude Medical Cardiology Division Incorporated

Method and system to automatically assign map points to anatomical segments and determine mechanical activation time  
Patent #
US 10,285,647 B2
Filed 05/05/2014

Current Assignee
Pacesetter Incorporated

Sponsoring Entity
Pacesetter Incorporated

Three dimensional image scan for vehicle  
Patent #
US 10,573,012 B1
Filed 09/21/2017

Current Assignee
Allstate Insurance Company

Sponsoring Entity
Allstate Insurance Company

Interactive imageguided surgical system for displaying images corresponding to the placement of a surgical tool or the like  
Patent #
US 5,230,338 A
Filed 04/22/1992

Current Assignee
MARKER LLC

Sponsoring Entity
Robert L. Galloway Jr, George S. Allen, Charles A. Edwards II, Martin R. Zink, Robert J. Maciunas

Three dimensional medical image display system  
Patent #
US 4,737,921 A
Filed 06/03/1985

Current Assignee
Picker International Inc.

Sponsoring Entity
DYNAMIC DIGITAL DISPLAYS INC. A PA. CORP.

Method and system for storing and communicating medical image data  
Patent #
US 4,958,283 A
Filed 07/08/1988

Current Assignee
Toshiba Corporation

Sponsoring Entity
Toshiba Corporation

Synthesized objectoriented entityrelationship (SOOER) model for coupled knowledgebase/database of image retrieval expert system (IRES)  
Patent #
US 5,615,112 A
Filed 01/29/1993

Current Assignee
Arizona Board of Regents, Toshiba Corporation

Sponsoring Entity
Arizona Board of Regents, Toshiba Corporation

Medical imaging system for displaying, manipulating and analyzing threedimensional images  
Patent #
US 6,175,655 B1
Filed 09/19/1996

Current Assignee
3D Medical Imaging Systems LLC

Sponsoring Entity
Integrated Medical Systems Incorporated

Interactive imageguided surgical system  
Patent #
US 5,142,930 A
Filed 03/29/1991

Current Assignee
MARKER LLC

Sponsoring Entity
Robert L. Galloway Jr, George S. Allen, Charles A. Edwards II, Martin R. Zink, Robert J. Maciunas

Separable image warping methods and systems using spatial lookup tables  
Patent #
US 5,204,944 A
Filed 07/28/1989

Current Assignee
Trustees Of Columbia University In The City Of New York

Sponsoring Entity
Trustees Of Columbia University In The City Of New York

Automatic analysis in virtual endoscopy  
Patent #
US 6,366,800 B1
Filed 04/23/1999

Current Assignee
Wake Forest University Health Sciences

Sponsoring Entity
Wake Forest University Health Sciences

Multiscale characterization and analysis of shapes  
Patent #
US 6,393,159 B1
Filed 06/29/1999

Current Assignee
Regents of the University of California

Sponsoring Entity
Regents of the University of California

Method and system for detection of lesions in medical images  
Patent #
US 6,185,320 B1
Filed 12/01/1997

Current Assignee
ARCH Development Corporation

Sponsoring Entity
ARCH Development Corporation

Methods and apparatus for performing sampling based synthesis of threedimensional geometric models  
Patent #
US 6,278,457 B1
Filed 01/15/1998

Current Assignee
GlobalFoundries Inc.

Sponsoring Entity
International Business Machines Corporation

Global constrained parameterization of triangulated surfaces  
Patent #
US 6,300,958 B1
Filed 07/17/1998

Current Assignee
TSURF S.A.

Sponsoring Entity
TSURF S.A.

Method and system for the assessment of tumor extent in magnetic resonance images  
Patent #
US 6,112,112 A
Filed 09/18/1998

Current Assignee
ARCH Development Corporation

Sponsoring Entity
ARCH Development Corporation

Computerassisted surgical method  
Patent #
US 5,871,018 A
Filed 06/06/1997

Current Assignee
Scott L. Delp, Stulberg S. David, Arthur Y. Wong, Loan J. Peter, Craig B. Robinson

Sponsoring Entity
Scott L. Delp, Stulberg S. David, Arthur Y. Wong, Loan J. Peter, Craig B. Robinson

Registration process for myocardial images  
Patent #
US 5,970,182 A
Filed 11/15/1995

Current Assignee
FOCUS IMAGING S.A.

Sponsoring Entity
FOCUS IMAGING S.A.

Method of forming a template of an image of an object for use in the recognition of the object  
Patent #
US 5,978,507 A
Filed 08/09/1994

Current Assignee
British Telecommunications BT Group Plc

Sponsoring Entity
British Telecommunications BT Group Plc

System and method of registration of threedimensional data sets  
Patent #
US 5,999,840 A
Filed 08/30/1995

Current Assignee
Brigham and Womens Hospital Incorporated

Sponsoring Entity
Brigham and Womens Hospital Incorporated

Method and apparatus for composing digital medical imagery  
Patent #
US 5,734,915 A
Filed 10/31/1994

Current Assignee
Eastman Kodak Company

Sponsoring Entity
Eastman Kodak Company

Computer based multimedia medical database management system and user interface  
Patent #
US 5,740,428 A
Filed 02/07/1995

Current Assignee
Merge Healthcare Incorporated

Sponsoring Entity
Merge Technologies Incorporated

Apparatus and method for photogrammetric surgical localization  
Patent #
US 5,603,318 A
Filed 10/29/1993

Current Assignee
Sofamor Danek Holdings Inc

Sponsoring Entity
University of Utah Research Foundation

Apparatus for determining the position and orientation of an invasive portion of a probe inside a threedimensional body  
Patent #
US 5,622,170 A
Filed 10/04/1994

Current Assignee
IMAGE GUIDED TECHNOLOGIES INC.

Sponsoring Entity
IMAGE GUIDED TECHNOLOGIES INC.

Imaging system having interactive medical instruments and methods  
Patent #
US 5,617,857 A
Filed 06/06/1995

Current Assignee
IMAGE GUIDED TECHNOLOGIES INC.

Sponsoring Entity
IMAGE GUIDED TECHNOLOGIES INC.

Registration of volumetric images which are relatively elastically deformed by matching surfaces  
Patent #
US 5,633,951 A
Filed 06/07/1995

Current Assignee
US Philips Corporation

Sponsoring Entity
US Philips Corporation

Method and apparatus for guiding an instrument to a target  
Patent #
US 5,638,819 A
Filed 08/29/1995

Current Assignee
Mark L. Manwaring, Kim H. Manwaring

Sponsoring Entity
Mark L. Manwaring, Kim H. Manwaring

Magnetic resonance imaging method and apparatus for guiding invasive therapy  
Patent #
US 5,647,361 A
Filed 03/01/1993

Current Assignee
Fonar Corporation

Sponsoring Entity
Fonar Corporation

Process of stereotactic optical navigation  
Patent #
US 5,662,111 A
Filed 05/16/1995

Current Assignee
INTEGRA RADIONICS INC.

Sponsoring Entity
Eric R. Cosman

Method and system for flexibly organizing, recording, and displaying medical patient care information using fields in a flowsheet  
Patent #
US 5,682,526 A
Filed 07/20/1995

Current Assignee
Uniloc Luxembourg S. r.l. fka Uniloc Luxembourg S.A.

Sponsoring Entity
Spacelabs Medical Incorporated

Method of searching for points of closest approach, and preprocessing method therefor  
Patent #
US 5,675,720 A
Filed 09/19/1996

Current Assignee
Fujitsu Limited

Sponsoring Entity
Fujitsu Limited

Computerassisted surgical system  
Patent #
US 5,682,886 A
Filed 12/26/1995

Current Assignee
CAST SPA

Sponsoring Entity
CAST SPA

Method for automatically registering a document having a plurality of pages  
Patent #
US 5,483,606 A
Filed 01/03/1994

Current Assignee
Xerox Corporation

Sponsoring Entity
Xerox Corporation

Digital data registration and differencing compression system  
Patent #
US 5,490,221 A
Filed 02/28/1992

Current Assignee
United States Of America As Represented By The Administrator Of The National Aeronautics And Space Administration, United States Incorporated

Sponsoring Entity
United States Of America As Represented By The Administrator Of The National Aeronautics And Space Administration, United States Incorporated

Process and device for the reproducible optical representation of a surgical operation  
Patent #
US 5,494,034 A
Filed 06/15/1994

Current Assignee
Georg Schlondorff

Sponsoring Entity
Georg Schlondorff

Stereotaxy wand and tool guide  
Patent #
US 5,517,990 A
Filed 04/08/1994

Current Assignee
Cleveland Clinic Foundation

Sponsoring Entity
Cleveland Clinic Foundation

Method and apparatus for representing a work area in a threedimensional structure  
Patent #
US 5,515,160 A
Filed 09/09/1994

Current Assignee
Aesculap AG

Sponsoring Entity
Aesculap AG

Coordinate measurement machine having a probe head and an electronic system for processing probe signals  
Patent #
US 5,526,576 A
Filed 09/12/1994

Current Assignee
Carl Zeiss Vision International GMBH

Sponsoring Entity
CarlZeissStiftung

System and method of registration of threedimensional data sets including anatomical body data  
Patent #
US 5,531,520 A
Filed 09/01/1994

Current Assignee
Engility Corp.

Sponsoring Entity
TASC Incorporated

Imaging device and method  
Patent #
US 5,531,227 A
Filed 01/28/1994

Current Assignee
SCHNEIDER MEDICAL TECHNOLOGIES INC.

Sponsoring Entity
SCHNEIDER MEDICAL TECHNOLOGIES INC.

Biomedical imaging and analysis  
Patent #
US 5,568,384 A
Filed 10/13/1992

Current Assignee
Intellectual Ventures Holding 15 LLC

Sponsoring Entity
MAYO MEDICAL VENTURES

Method for autonomous image registration  
Patent #
US 5,581,638 A
Filed 02/28/1995

Current Assignee
Raytheon Company

Sponsoring Entity
Raytheon ESystems Inc.

Method and apparatus for acquiring complete radon data for exactly reconstructing a three dimensional computerized tomography image of a portion of an object irradiated by a cone beam source  
Patent #
US 5,383,119 A
Filed 07/03/1991

Current Assignee
General Electric Company

Sponsoring Entity
General Electric Company

System for indicating the position of a surgical probe within a head on an image of the head  
Patent #
US 5,383,454 A
Filed 07/02/1992

Current Assignee
St. Louis University

Sponsoring Entity
St. Louis University

Apparatus and method for photogrammetric surgical localization  
Patent #
US 5,389,101 A
Filed 04/21/1992

Current Assignee
Sofamor Danek Holdings Inc

Sponsoring Entity
University of Utah Research Foundation

Method and apparatus for constructing threedimensional surface shading image display  
Patent #
US 5,402,337 A
Filed 07/24/1992

Current Assignee
Toshiba Corporation

Sponsoring Entity
Toshiba Corporation

Method and apparatus for video presentation from scanner imaging sources  
Patent #
US 5,398,684 A
Filed 03/31/1992

Current Assignee
Tyrone L. Hardy

Sponsoring Entity
Tyrone L. Hardy

Method for displaying an image of a portion of the inside of a threedimensional physical structure  
Patent #
US 5,412,763 A
Filed 05/24/1991

Current Assignee
General Electric Company

Sponsoring Entity
General Electric Company

Method and apparatus for generating realistic images using a discrete representation  
Patent #
US 5,442,733 A
Filed 03/20/1992

Current Assignee
The Research Foundation for The State University of New York

Sponsoring Entity
The Research Foundation for The State University of New York

Automated system and a method for organizing, presenting, and manipulating medical images  
Patent #
US 5,452,416 A
Filed 12/30/1992

Current Assignee
Merge Healthcare Solutions Incorporated

Sponsoring Entity
DR Systems Incorporated

Method for determining the position of an organ  
Patent #
US 5,447,154 A
Filed 07/30/1993

Current Assignee
Universit Joseph Fourier

Sponsoring Entity
Universit Joseph Fourier

Method for constructing a three dimensional scanning trajectory capable of acquiring a complete set of radon data for exact image reconstruction of an object irradiated by a cone beam source  
Patent #
US 5,463,721 A
Filed 07/29/1991

Current Assignee
General Electric Company

Sponsoring Entity
General Electric Company

Report generating system  
Patent #
US 5,465,378 A
Filed 10/12/1993

Current Assignee
DITO SOLUTIONS L.L.C.

Sponsoring Entity
Compuspeak Inc.

Threedimensional impedance imaging processes  
Patent #
US 5,284,142 A
Filed 12/16/1991

Current Assignee
Rensselaer Polytechnic Institute

Sponsoring Entity
Rensselaer Polytechnic Institute

Teleradiology system  
Patent #
US 5,291,401 A
Filed 11/15/1991

Current Assignee
Internal Revenue Service

Sponsoring Entity
Internal Revenue Service

Locating target in human body  
Patent #
US 5,295,483 A
Filed 10/11/1991

Current Assignee
Northern Digital Inc.

Sponsoring Entity
Mark T. Horbal, Christopher Nowacki

Imagedirected robotic system for precise robotic surgery including redundant consistency checking  
Patent #
US 5,299,288 A
Filed 09/18/1991

Current Assignee
Regents of the University of California, International Business Machines Corporation

Sponsoring Entity
Regents of the University of California, International Business Machines Corporation

Method and apparatus for determining the alignment of an object  
Patent #
US 5,295,200 A
Filed 12/03/1992

Current Assignee
Board of Regents of the University of Texas System

Sponsoring Entity
Board of Regents of the University of Texas System

Apparatus and method for spatially positioning images  
Patent #
US 5,291,889 A
Filed 05/23/1991

Current Assignee
VANGUARD IMAGING LTD. A CORP. OF DELAWARE

Sponsoring Entity
VANGUARD IMAGING LTD. A CORP. OF DELAWARE

Computeraided surgery apparatus  
Patent #
US 5,305,203 A
Filed 10/02/1990

Current Assignee
Faro Technologies Incorporated

Sponsoring Entity
Xenon Research Inc.

Frameless stereotaxy system  
Patent #
US 5,309,913 A
Filed 11/30/1992

Current Assignee
Cleveland Clinic Foundation

Sponsoring Entity
Cleveland Clinic Foundation

Threedimensional reprojected image forming apparatus  
Patent #
US 5,309,356 A
Filed 04/02/1990

Current Assignee
Toshiba Corporation

Sponsoring Entity
Toshiba Corporation

Optical image analyzer using optical correlation and optoelectronic feedback  
Patent #
US 5,323,472 A
Filed 02/11/1993

Current Assignee
University of California San Diego, Regents of the University of California

Sponsoring Entity
The Boeing Co.

Three dimensional image reconstruction for a positron emission tomograph  
Patent #
US 5,331,553 A
Filed 04/15/1992

Current Assignee
Ugm Medical Systems Inc.

Sponsoring Entity
Ugm Medical Systems Inc.

Generalized shape autocorrelation for shape acquisition and recognition  
Patent #
US 5,351,310 A
Filed 12/08/1993

Current Assignee
International Business Machines Corporation

Sponsoring Entity
International Business Machines Corporation

Threedimensional impedance imaging process  
Patent #
US 5,351,697 A
Filed 09/29/1993

Current Assignee
Rensselaer Polytechnic Institute

Sponsoring Entity
Rensselaer Polytechnic Institute

Surgical microscope for conducting computersupported stereotactic microsurgery and a method for operating the same  
Patent #
US 5,359,417 A
Filed 10/15/1992

Current Assignee
CarlZeissStiftung

Sponsoring Entity
CarlZeissStiftung

Apparatus and method for displaying threedimensional image and ultrasonic diagnosis apparatus using the same  
Patent #
US 5,353,220 A
Filed 03/19/1993

Current Assignee
Hitachi Medical Corporation

Sponsoring Entity
Hitachi Medical Corporation

Concurrent display and adjustment of 3D projection, coronal slice, sagittal slice, and transverse slice images  
Patent #
US 5,371,778 A
Filed 11/29/1991

Current Assignee
Picker International Inc.

Sponsoring Entity
Picker International Inc.

Method for implanting a fiducial implant into a patient  
Patent #
US 5,178,164 A
Filed 03/29/1991

Current Assignee
MARKER LLC

Sponsoring Entity
George S. Allen

Process and device for the reproducible optical representation of a surgical operation  
Patent #
US 5,186,174 A
Filed 01/29/1990

Current Assignee
Prof. Dr. Schlondorff George

Sponsoring Entity
G. M. PFAFF

Method and apparatus for threedimensional noncontact shape sensing  
Patent #
US 5,198,877 A
Filed 10/15/1990

Current Assignee
Northern Digital Inc.

Sponsoring Entity
PIXSYS INC. A CORP. OF COLORADO

Locating target in human body  
Patent #
US 5,197,476 A
Filed 10/11/1991

Current Assignee
Northern Digital Inc.

Sponsoring Entity
Christopher Nowacki, Mart T. Horbal

Apparatus for and method of performing stereotaxic surgery  
Patent #
US 5,207,223 A
Filed 10/19/1990

Current Assignee
Accuray Incorporated

Sponsoring Entity
John R. Adler

Method and apparatus for imaging the anatomy  
Patent #
US 5,222,499 A
Filed 03/26/1992

Current Assignee
MARKER LLC

Sponsoring Entity
John M. Fitzpatrick, George S. Allen, Venkateswara R. Mandava, Hsuan Chang, Robert J. Maciunas

3dimensional image display apparatus capable of displaying a 3D image by manipulating a positioning encoder  
Patent #
US 5,229,935 A
Filed 07/25/1990

Current Assignee
Toshiba Corporation

Sponsoring Entity
Toshiba Corporation

Method of identifying and archiving medical images  
Patent #
US 5,241,472 A
Filed 10/15/1990

Current Assignee
University of Pittsburgh  of The Commonwealth System of Higher Education

Sponsoring Entity
University of Pittsburgh  of The Commonwealth System of Higher Education

Computeraided surgery apparatus  
Patent #
US 5,251,127 A
Filed 07/31/1990

Current Assignee
Faro Technologies Incorporated

Sponsoring Entity
Faro Technologies Incorporated

Precision bone alignment  
Patent #
US 5,249,581 A
Filed 07/15/1991

Current Assignee
Northern Digital Inc.

Sponsoring Entity
Mark T. Horbal, Christopher Nowacki

Threedimensional mammal anatomy imaging system and method  
Patent #
US 5,261,404 A
Filed 07/08/1991

Current Assignee
Peter R. Mick, Mark Savet

Sponsoring Entity
Peter R. Mick, Mark Savet

Medical threedimensional locating apparatus  
Patent #
US 5,257,998 A
Filed 06/30/1992

Current Assignee
Mitaka Kohki Co. Ltd., Takaaki Takizawa

Sponsoring Entity
Mitaka Kohki Co. Ltd., Takaaki Takizawa

Medical image data managing system  
Patent #
US 5,272,625 A
Filed 05/16/1991

Current Assignee
Toshiba Corporation

Sponsoring Entity
Toshiba Corporation

Method and apparatus for realtime navigation assist in interventional radiological procedures  
Patent #
US 5,274,551 A
Filed 11/29/1991

Current Assignee
General Electric Company

Sponsoring Entity
General Electric Company

Quick threedimensional display  
Patent #
US 5,079,699 A
Filed 08/09/1989

Current Assignee
Picker International Inc.

Sponsoring Entity
Picker International Inc.

Imaging device  aided robotic stereotaxis system  
Patent #
US 5,078,140 A
Filed 09/23/1986

Current Assignee
Yik S. Kwoh

Sponsoring Entity
Yik S. Kwoh

Imagedirected robotic system for precise robotic surgery including redundant consistency checking  
Patent #
US 5,086,401 A
Filed 05/11/1990

Current Assignee
International Business Machines Corporation

Sponsoring Entity
International Business Machines Corporation

Method and apparatus for video presentation from a variety of scanner imaging sources  
Patent #
US 5,099,846 A
Filed 12/23/1988

Current Assignee
Tyrone L. Hardy

Sponsoring Entity
Tyrone L. Hardy

Apparatus for imaging the anatomy  
Patent #
US 5,094,241 A
Filed 01/19/1990

Current Assignee
MARKER LLC

Sponsoring Entity
George S. Allen

Apparatus for imaging the anatomy  
Patent #
US 5,119,817 A
Filed 01/19/1990

Current Assignee
MARKER LLC

Sponsoring Entity
George S. Allen

Reconstructing two and three dimensional images by two and three dimensional Fourier transforms in an MRI system  
Patent #
US 5,166,875 A
Filed 11/17/1989

Current Assignee
Toshiba Corporation

Sponsoring Entity
Toshiba Corporation

Method and apparatus for the simultaneous display and correlation of independently generated images  
Patent #
US 4,987,412 A
Filed 08/25/1988

Current Assignee
United States Department of Energy

Sponsoring Entity
United States Department of Energy

Method and apparatus for providing related images over time of a portion of the anatomy using fiducial implants  
Patent #
US 4,991,579 A
Filed 11/10/1987

Current Assignee
MARKER LLC

Sponsoring Entity
George S. Allen

System and method for remote presentation of diagnostic image information  
Patent #
US 5,005,126 A
Filed 05/31/1988

Current Assignee
Prevail Inc.

Sponsoring Entity
Prevail Inc.

Method and apparatus for imaging the anatomy  
Patent #
US 5,016,639 A
Filed 02/13/1990

Current Assignee
George S. Allen

Sponsoring Entity
George S. Allen

Confirmed boundary pattern matching  
Patent #
US 5,027,422 A
Filed 08/29/1988

Current Assignee
RAYTHEON COMPANY LEXINGTON MASSACHUSETTS 02173 A CORP. OF DE

Sponsoring Entity
RAYTHEON COMPANY LEXINGTON MASSACHUSETTS 02173 A CORP. OF DE

Therapeutic apparatus  
Patent #
US 5,039,867 A
Filed 01/08/1991

Current Assignee
Mitsubishi Electric Corporation

Sponsoring Entity
Mitsubishi Electric Corporation

System for indicating a position to be operated in a patient's body  
Patent #
US 5,050,608 A
Filed 09/04/1990

Current Assignee
MIZUHO IKAKOGYO CO. LTD.

Sponsoring Entity
Medirand Inc.

Method for the automatic resetting of images  
Patent #
US 5,048,103 A
Filed 03/16/1990

Current Assignee
General Electric Company

Sponsoring Entity
General Electric Company

Method and apparatus for stone localization using ultrasound imaging  
Patent #
US 4,896,673 A
Filed 07/15/1988

Current Assignee
MEDSTONE INTERNATIONAL INC.

Sponsoring Entity
MEDSTONE INTERNATIONAL INC.

Method and apparatus for providing related images over time of a portion of the anatomy using at least four fiducial implants  
Patent #
US 4,945,914 A
Filed 07/18/1988

Current Assignee
MARKER LLC

Sponsoring Entity
George S. Allen

System for the simultaneous display of two or more internal surfaces within a solid object  
Patent #
US 4,821,213 A
Filed 12/19/1986

Current Assignee
General Electric Company

Sponsoring Entity
General Electric Company

Method and apparatus for manufacturing a modified, threedimensional reproduction of a soft, deformable object  
Patent #
US 4,821,200 A
Filed 04/02/1987

Current Assignee
JONKOPINGS LANS LANDSTING A CORP. OF SWEDEN

Sponsoring Entity
JONKOPINGS LANS LANDSTING A CORP. OF SWEDEN

Threedimensional image processing apparatus  
Patent #
US 4,835,688 A
Filed 01/26/1988

Current Assignee
Toshiba Corporation

Sponsoring Entity
Toshiba Corporation

Mandibular motion monitoring system  
Patent #
US 4,836,778 A
Filed 05/26/1987

Current Assignee
Vexcel Corporation

Sponsoring Entity
Vexcel Corporation

Method of displaying internal surfaces of threedimensional medical images  
Patent #
US 4,879,668 A
Filed 12/19/1986

Current Assignee
General Electric Company

Sponsoring Entity
General Electric Company

Reference display systems for superimposing a tomagraphic image onto the focal plane of an operating microscope  
Patent #
US 4,722,056 A
Filed 02/18/1986

Current Assignee
The Trustees of Dartmounth College

Sponsoring Entity
The Trustees of Dartmounth College

Computer tomography assisted stereotactic surgery system and method  
Patent #
US 4,791,934 A
Filed 08/07/1986

Current Assignee
Brainlab AG

Sponsoring Entity
Picker International Inc.

Image registration method  
Patent #
US 4,644,582 A
Filed 01/24/1984

Current Assignee
Hitachi Medical Corporation, Hitachi Ltd.

Sponsoring Entity
Hitachi Medical Corporation, Hitachi Ltd.

Device for measuring relative jaw positions and movements  
Patent #
US 4,673,352 A
Filed 01/02/1986

Current Assignee
Markus Hansen

Sponsoring Entity
Markus Hansen

Measuring system employing a measuring method based on the triangulation principle for the noncontact measurement of a distance from the surface of a contoured object to a reference level. _  
Patent #
US 4,701,049 A
Filed 06/19/1984

Current Assignee
B.V. Optische Industrie De Oude Delft

Sponsoring Entity
B.V. Optische Industrie De Oude Delft

Method and apparatus for stereotaxic placement of probes in the body utilizing CT scanner localization  
Patent #
US 4,583,538 A
Filed 05/04/1984

Current Assignee
Sherwood Services AG

Sponsoring Entity
Eric R. Cosman, Gary M. Onik, Trent H. Wells

Image correspondence techniques using serial neighborhood processing  
Patent #
US 4,590,607 A
Filed 07/16/1985

Current Assignee
ERIM INTERNATIONAL INC.

Sponsoring Entity
ENVIRONMENTAL RESEARCH INSTITUTE OF MICHIGAN THE A CORP OF MICHIGAN

System using computed tomography as for selective body treatment  
Patent #
US 4,608,977 A
Filed 12/20/1982

Current Assignee
Sherwood Services AG

Sponsoring Entity
Russell A. Brown

Method and apparatus for monitoring of positions of patients and/or radiation units  
Patent #
US 4,262,306 A
Filed 11/21/1979

Current Assignee
Karlheinz Renner

Sponsoring Entity
Karlheinz Renner

17 Claims
 1. A method for registering a template image with a target image, the template image containing a plurality of points and the target image containing a plurality of points, comprising:
defining at least one manifold landmark point in said template image;
identifying at least one point in said target image corresponding to said defined at least one manifold landmark point;
selecting a coordinate frame suitable for spherical geometries;
computing a large deformation transform using said selected coordinate frame, at least one manifold landmark transformation operator, and at least one manifold landmark transformation boundary value, said large deformation transform relating said at least one manifold landmark point in said template image to said corresponding at least one point in said target image; and
registering said template image with said target image using said large deformation transform in the selected coordinate frame.  View Dependent Claims (2, 3, 4, 5, 6)
 7. A method for registering a template image with a target image, comprising:
selecting a coordinate frame which is twodimensional only and which is suitable for spherical geometries; and
registering the target and template image using a large deformation transform in the selected coordinate frame.  View Dependent Claims (8, 9, 10, 11)
 12. A method for registering a template image with a target image, wherein the template image contains a plurality of points and the target image contains a plurality of points, comprising:
selecting a coordinate frame suitable for spherical geometries;
computing a first large deformation transform using at least one manifold landmark transformation operator relating at least one manifold landmark point in said template image to at least one corresponding point in said target image;
computing a second large deformation transform using at least one image transformation operator relating said template image to said target image;
fusing said first and second large deformation transforms; and
registering said template image with said target image using said fused large deformation transforms in said selected coordinate frame.
 13. An apparatus for registering a template image with a target image, the template image containing a plurality of points and the target image containing a plurality of points, comprising:
means for defining at least one manifold landmark point in said template image;
means for identifying at least one point in said target image corresponding to said defined at least one manifold landmark point;
means for selecting a coordinate frame suitable for spherical geometries;
means for computing a large deformation transform using said selected coordinate frame, at least one manifold landmark transformation operator, and at least one manifold landmark transformation boundary value, said large deformation transform relating said at least one manifold landmark point in said template image to said corresponding at least one point in said target image; and
means for registering said template image with said target image using said large deformation transform in the selected coordinate frame.
 14. An apparatus for registering a template image with a target image, the template image containing a plurality of points and the target image containing a plurality of points, comprising:
means for selecting a coordinate frame which is twodimensional only and which is suitable for spherical geometries; and
means for registering the target and template image using a large deformation transform in the selected coordinate frame.
 15. An apparatus for registering a template image with a target image, the template image containing a plurality of points and the target image containing a plurality of points, comprising:
a coordinate frame which is twodimensional only and which is suitable for spherical geometries; and
a registration processor for registering the target and template image using a large deformation transform in the selected coordinate frame.
 16. An apparatus for registering a template image with a target image, wherein the template image contains a plurality of points and the target image contains a plurality of points, comprising:
a pointing device for defining at least one manifold landmark point in said template image and for identifying at least one point in said target image corresponding to said at least one defined manifold landmark point;
a coordinate frame selector for selecting a coordinate frame suitable for spherical geometries;
a first data processing unit for computing a first large deformation transform using a manifold landmark transformation operator relating said at least one manifold landmark point in said template image to said corresponding at least one point in said target image;
a distance quantifier for defining a distance between said target image and said template image;
a second data processing unit for computing a second large deformation transform using an image transformation operator relating said template image to said target image; and
a third data processing unit for fusing said first and second large deformation transforms; and
a fourth data processing unit for registering said template image with said target image using said fused large deformation transforms in said coordinate frame suitable for spherical geometries.
 17. A computer program product for use in a computer adapted for registering a template image with a target image, the computer program product comprising a computer readable medium for storing computer readable code means, which when executed by the computer, enables the computer to register a template image with a target image, and wherein the computer readable code means includes computer readable instructions for causing the computer to execute a method comprising:
selecting a coordinate frame which is twodimensional only and which is suitable for spherical geometries; and
registering the target and template image using a large deformation transform in the selected coordinate frame.
1 Specification
This is a continuationinpart of prior U.S. patent application Ser. No. 09/186,359 filed Nov. 5, 1998, now U.S. Pat. No. 6,226,418. This continuationinpart claims priority to U.S. patent application Ser. No. 09/186,359 and incorporates the same by reference herein. This continuationinpart also claims priority to U.S. provisional application 60/155,141 filed Sep. 22, 1999 (now expired) and incorporates the same by reference herein.
This work was supported in part by the following U.S. Government grants: NIH grants RR01380 and R01MH5213801A1; ARO grant DAAL0386K0110; and NSF grant BIR9424 264. The U.S. Government may have certain rights in the invention.
The present invention relates to image processing systems and methods, and more particularly to image registration systems that combine two or more images into a composite image; in particular the fusion of anatomical manifoldbased knowledge with volume imagery via large deformation mapping which supports both kinds of information simultaneously, as well as individually, and which can be implemented on a rapid convolution FFT based computer system.
Image registration involves combining two or more images, or selected points from the images, to produce a composite image containing data from each of the registered images. During registration, a transformation is computed that maps related points among the combined images so that points defining the same structure in each of the combined images are correlated in the composite image.
Currently, practitioners follow two different registration techniques. The first requires that an individual with expertise in the structure of the object represented in the images label a set of landmarks in each of the images that are to be registered. For example, when registering two MRI images of different axial slices of a human head, a physician may label points, or a contour surrounding these points, corresponding to the cerebellum in two images. The two images are then registered by relying on a known relationship among the landmarks in the two brain images. The mathematics underlying this registration process is known as small deformation multitarget registration.
In the previous example of two brain images being registered, using a purely operatordriven approach, a set of N landmarks identified by the physician, represented by x_{i}, where i=1 . . . N, are defined within the two brain coordinate systems. A mapping relationship, mapping the N points selected in one image to the corresponding N points in the other image, is defined by the equation u(x_{i})=k_{i}, where i=1 . . . N. Each of the coefficients, k_{i}, is assumed known.
The mapping relationship u(x) is extended from the set of N landmark points to the continuum using a linear quadratic form regularization optimization of the equation:
subject to the boundary constraints u(x_{i})=k_{i′}. The operator L is a linear differential operator. This linear optimization problem has a closed form solution. Selecting L=α∇^{2}+β∇(∇·) gives rise to small deformation elasticity.
For a description of small deformation elasticity see S. Timoshenko, Theory of Elasticity, McGrawHill, 1934 (hereinafter referred to as Timoshenko) and R. L. Bisplinghoff, J. W. Marr, and T. H. H. Pian, Statistics of Deformable Solids, Dover Publications, Inc., 1965 (hereinafter referred to as Bisplinghoff). Selecting L=∇^{2 }gives rise to a membrane or Laplacian model. Others have used this operator in their work, see e.g., Amit, U. Grenander, and M. Piccioni, “Structural image restoration through deformable templates,” J. American Statistical Association. 86(414):376387, June 1991, (hereinafter referred to as Amit) and R. Szeliski, Bayesian Modeling of Uncertainty in LowLevel Vision, Kluwer Academic Publisher, Boston, 1989 (hereinafter referred to as Szeliski) (also describing a biharmonic approach). Selecting L=∇^{4 }gives a spline or biharmonic registration method. For examples of applications using this operator see Grace Wahba, “Spline Models for Observational Data,” Regional Conference Series in Applied Mathematics. SIAM, 1990, (hereinafter referred to as Whaba) and F. L. Bookstein, The Measurement of Biological Shape and Shape Change, volume 24, SpringerVerlag: Lecture Notes in Biomathematics, New York, 1978 (hereinafter referred to as Bookstein).
The second currentlypracticed technique for image registration uses the mathematics of small deformation multitarget registration and is purely image data driven. Here, volume based imagery is generated of the two targets from which a coordinate system transformation is constructed. Using this approach, a distance measure, represented by the expression D(u), represents the distance between a template T(x) and a target image S(x). The optimization equation guiding the registration of the two images using a distance measure is:
The distance measure D(u) measuring the disparity between imagery has various forms, e.g., the Gaussian squared error distance ∫T(h(x))−S(x)^{2}dx, a correlation distance, or a Kullback Liebler distance. Registration of the two images requires finding a mapping that minimizes this distance.
Other fusion approaches involve small deformation mapping coordinates x ε Ω of one set of imagery to a second set of imagery. Other techniques include the mapping of predefined landmarks and imagery, both taken separately such as in the work of Bookstein, or fused as covered via the approach developed by MillerJoshiChristensenGrenander the '"'"'212 patent described in U.S. Pat. No. 6,009,212 (hereinafter “the '"'"'212 patent”) herein incorporated by reference mapping coordinates x ε Ω of one target to a second target. The existing state of the art for small deformation matching can be stated as follows:
Small Deformation Matching: Construct h(x)=x−u(x) according to the minimization of the distance D(u) between the template and target imagery subject to the smoothness penalty defined by the linear differential operator L:
The distance measure changes depending upon whether landmarks or imagery are being matched.
1. Landmarks alone. The first approach is purely operator driven in which a set of point landmarks x_{i}, i=1, . . . N are defined within the two brain coordinate systems by, for example, an anatomical expert, or automated system from which the mapping is assumed known: u(x_{i})=k_{i}, i=1, . . . , N. The field u(x) specifying the mapping h is extended from the set of points {x_{i}} identified in the target to the points {y_{i}} measured with Gaussian error covariances Σ_{i}:
Here (·)^{τ }denotes transpose of the 3×1 real vectors, and L is a linear differential operator giving rise to small deformation elasticity (see Timoshenko and Bisplinghoff), the membrane of Laplacian model (see Amit and Szeliski), biharmonic (see Szeliski), and many of the spline methods (see Wahba and Bookstein). This is a linear optimization problem with closed form solution.
2. The second approach is purely volume image data driven, in which the volume based imagery is generated of the two targets from which the coordinate system transformation is constructed. A distance measure between the two images being registered I_{0}, I_{1 }is defined as D(u)=∫I_{0}(x−u(x))−I_{1}(x)^{2}dx. The corresponding optimization is:
The data function D(u) measures the disparity between imagery and has various forms. Other distances are used besides the Gaussian squared error distance, including correlation distance, Kullback Liebler distance, and others.
3. The algorithm for the transformation of imagery I_{0 }into imagery I_{1 }has landmark and volume imagery fused in the small deformation setting as in the '"'"'212 patent. Both sources of information are combined into the small deformation registration:
Although small deformation methods provide geometrically meaningful deformations under conditions where the imagery being matched are small, linear, or affine changes from one image to the other. Small deformation mapping does not allow the automatic calculation of tangents, curvature, surface areas, and geometric properties of the imagery. To illustrate the mapping problem FIG. 9 shows an oval template image with several landmarks highlighted. FIG. 10 shows a target image that is greatly deformed from the template image. The target image is a largely deformed oval that has been twisted. FIG. 11 shows the results of image matching when the four corners are fixed, using small deformation methods based on static quadratic form regularization. These figures illustrate the distortion which occurs with small deformation linear mapping when used with landmark points which define a motion corresponding to large deformation. As can be seen in FIG. 11, landmarks defined in the template image often map to more than one corresponding point in the target image.
Large deformation mapping produces maps for image registration in which the goal is to find the onetoone, onto, invertible, differentiable maps h (henceforth termed diffeomorphisms) from the coordinates x ε Ω of one target to a second target under the mapping
To accommodate very fine variations in anatomy the diffeomorphic transformations constructed are of high dimensions having, for example a dimension greater than 12 of the Affine transform upto the order of the number of voxels in the volume. A transformation is diffeomorphic if the transformation from the template to the target is onetoone, onto, invertible, and both the transformation and it'"'"'s inverse are differentiable. A transformation is said to be onetoone if no two distinct points in the template are mapped to the same point in the target. A transformation is said to be onto if every point in the target is mapped from a point in the template. The importance of generating diffeomorphisms is that tangents, curvature, surface areas, and geometric properties of the imagery can be calculated automatically. FIG. 12 illustrates the image mapping illustrated in FIG. 11 using diffeomorphic transformation.
The present invention overcomes the limitations of the conventional techniques of image registration by providing a methodology which combines, or fuses, some aspects of techniques where an individual with expertise in the structure of the object represented in the images labels a set of landmarks in each image that are to be registered and techniques that use mathematics of small deformation multitarget registration, which is purely image data driven.
Additional features and advantages of the invention will be set forth in the description which follows, and in part, will be apparent from the description, or may be learned by practicing the invention. The embodiments and other advantages of the invention will be realized and obtained by the method and apparatus particularly pointed out in the written description and the claims hereof as well as in the appended drawings.
To achieve these and other advantages and in accordance with the purpose of the invention, as embodied and broadly described, an embodiment consistent with the present invention registers a template and a target image by defining a manifold landmark point in the template image, identifying a point in said target image corresponding to the defined manifold landmark point, and selecting a coordinate frame suitable for spherical geometries. According to an embodiment of the present invention, a large deformation transform is computed using the selected coordinate frame, a manifold landmark transformation operator, and at least one manifold landmark transformation boundary value. The large deformation transform relates the manifold landmark point in the template image to the corresponding point in the target image. The template and target image are registered using the large deformation transform in the selected coordinate frame. Another embodiment consistent with the present invention registers a template image with a target image by selecting a coordinate frame suitable for spherical geometries and registering the target and template image using a large deformation transform in the selected coordinate frame.
Both the foregoing general description and the following detailed description are exemplary and explanatory and are intended to provide further explanation of the invention as claimed.
The accompanying drawings provide a further understanding of the invention. They illustrate embodiments of the invention and, together with the description, explain the principles of the invention.
FIG. 1 is a target and template image of an axial section of a human head with 0dimensional manifolds;
FIG. 2 is schematic diagram illustrating an apparatus for registering images in accordance with the present invention;
FIG. 3 is a flow diagram illustrating the method of image registration according to the present invention;
FIG. 4 is a target and a template image with 1dimensional manifolds;
FIG. 5 is a target and a template image with 2dimensional manifolds;
FIG. 6 is a target and a template image with 3dimensional manifolds;
FIG. 7 is sequence of images illustrating registration of a template and target image; and
FIG. 8 is a flow diagram illustrating the computation of a fusing transform;
FIG. 9 is an oval template image which has landmark points selected and highlighted;
FIG. 10 is a deformed and distorted oval target image with corresponding landmark points highlighted and selected;
FIG. 11 is an image matching of the oval target and template images; and
FIG. 12 is an image matching using diffeomorphism.
1. Method for Image Registration Using Both Landmark Based knowledge and Image Data
A method and system is disclosed which registers images using both landmark based knowledge and image data. Reference will now be made in detail to the present preferred embodiment of the invention, examples of which are illustrated in the accompanying drawings.
To illustrate the principles of this invention, FIG. 1 shows two axial views of a human head. In this example, template image 100 contains points 102, 104, and 114 identifying structural points (0dimensional landmark manifolds) of interest in the template image. Target image 120 contains points 108, 110, 116, corresponding respectively to template image points 102, 104, 114, via vectors 106, 112, 118, respectively.
FIG. 2 shows apparatus to carry out the preferred embodiment of this invention. A medical imaging scanner 214 obtains the images show in FIG. 1 and stores them on a computer memory 206 which is connected to a computer central processing unit (CPU) 204. One of ordinary skill in the art will recognize that a parallel computer platform having multiple CPUs is also a suitable hardware platform for the present invention, including, but not limited to, massively parallel machines and workstations with multiple processors. Computer memory 206 can be directly connected to CPU 204, or this memory can be remotely connected through a communications network.
Registering images 100, 120 according to the present invention, unifies registration based on landmark deformations and image data transformation using a coarsetofine approach. In this approach, the highest dimensional transformation required during registration is computed from the solution of a sequence of lower dimensional problems driven by successive refinements. The method is based on information either provided by an operator, stored as defaults, or determined automatically about the various substructures of the template and the target, and varying degrees of knowledge about these substructures derived from anatomical imagery, acquired from modalities like CT, MRI, functional MRI, PET, ultrasound, SPECT, MEG, EEG, or cryosection.
Following this hierarchical approach, an operator, using pointing device 208, moves cursor 210 to select points 102, 104, 114 in FIG. 1, which are then displayed on a computer monitor 202 along with images 100, 120. Selected image points 102, 104, and 114 are 0dimensional manifold landmarks.
Once the operator selects manifold landmark points 102, 104, and 114 in template image 100, the operator identifies the corresponding template image points 108, 110, 116.
Once manifold landmark selection is complete, CPU 204 computes a first transform relating the manifold landmark points in template image 100 to their corresponding image points in target image 120. Next, a second CPU 204 transform is computed by fusing the first transform relating selected manifold landmark points with a distance measure relating all image points in both template image 100 and target image 120. The operator can select an equation for the distance measure several ways including, but not limited to, selecting an equation from a list using pointing device 208, entering into CPU 204 an equation using keyboard 212, or reading a default equation from memory 206. Registration is completed by CPU 204 applying the second computed transform to all points in the template image 100.
Although several of the registration steps are described as selections made by an operator, implementation of the present invention is not limited to manual selection. For example, the transforms, boundary values, region of interest, and distance measure can be defaults read from memory or determined automatically.
FIG. 3 illustrates the method of this invention in operation. First an operator defines a set of N manifold landmark points x_{i, }where i=1, . . . , N, represented by the variable M, in the template image (step 300). These points should correspond to points that are easy to identify in the target image.
Associated with each landmark point, x_{i}, in the template image, is a corresponding point y_{i }in the target image. The operator therefore next identifies the corresponding points, y_{i}, in the target image are identified (step 310). The nature of this process means that the corresponding points can only be identified within some degree of accuracy. This mapping between the template and target points can be specified with a resolution having a Gaussian error of variance σ^{2}.
If a transformation operator has not been designated, the operator can choose a manifold landmark transformation operator, L, for this transformation computation. In this embodiment, the Laplacian
is used for the operator L. Similarly, the operator can also select boundary values for the calculation corresponding to assumed boundary conditions, if these values have not been automatically determined or stored as default values. Here, infinite boundary conditions are assumed, producing the following equation for K, where K(x,x_{i}) is the Green'"'"'s function of a volume landmark transformation operator L^{2 }(assuming L is selfadjoint):
Using circulant boundary conditions instead of infinite boundary conditions provides and embodiment suitable for rapid computation. One of ordinary skill in the art will recognize that other operators can be used in place of the Laplacian operator, such operators include, but are not limited to, the biharmonic operator, linear elasticity operator, and other powers of these operators.
In addition, the operator may select a region of interest in the target image. Restricting the computation to a relatively small region of interest reduces both computation and storage requirements because transformation is computed only over a subregion of interest. It is also possible that in some applications the entire image is the desired region of interest. In other applications, there may be default regions of interest that are automatically identified.
The number of computations required is proportional to the number of points in the region of interest, so the computational savings equals the ratio of the total number of points in the image to the number of points in the region of interest. The data storage savings for an image with N points with a region of interest having M points is a factor of N/M. For example, for a volume image of 256×256×256 points with a region of interest of 128×128×128 points, the computation time and the data storage are reduced by a factor of eight.
In addition, performing the computation only over the region of interest makes it necessary only to store a subregion, providing a data storage savings for the template image, the target image, and the transform values.
Following the identification of template manifold landmark points and corresponding points in the target image, as well as selection of the manifold transformation operator, the boundary values, and the region of interest, CPU 204 computes a transform that embodies the mapping relationship between these two sets of points (step 350). This transform can be estimated using Bayesian optimization, using the following equation:
the minimizer, u, having the form
where A is a 3×3 matrix, b=[b_{1}, b_{2}, b_{3}] is a 3×1 vector, [β_{i1},β_{i2},β_{i3}] is a 3×1 weighting vector.
The foregoing steps of the image registration method provide a coarse matching of a template and a target image. Fine matching of the images requires using the full image data and the landmark information and involves selecting a distance measure by solving a synthesis equation that simultaneously maps selected image landmarks in the template and target images and matches all image points within a region of interest. An example of this synthesis equation is:
here the displacement field u is constrained to have the form
with the variables β_{i}, A, and b, computed at step 350 in FIG. 3. The operator L in equation (11) may be the same operator used in equation (9), or alternatively, another operator may be used with a different set of boundary conditions. The basis functions φ are the eigen functions of operators such as the Laplacian Lu=∇^{2}u, the biharmonic Lu=∇^{4}u, linear elasticity Lu=α∇^{2}u+(α+β)∇(∇.u), and powers of these operators L^{p }for p≧1.
One of ordinary skill in the art will recognize that there are many possible forms of the synthesis equation. For example, in the synthesis equation presented above, the distance measure in the first term measures the relative position of points in the target image with respect to points in the template image. Although this synthesis equation uses a quadratic distance measure, one of ordinary skill in the art will recognize that there are other suitable distance measures.
CPU 204 then computes a second or fusing transformation (Step 370) using the synthesis equation relating all points within a region of interest in the target image to all corresponding points in the template image. The synthesis equation is defined so that the resulting transform incorporates, or fuses, the mapping of manifold landmarks to corresponding target image points determined when calculating the first transform.
The computation using the synthesis equation is accomplished by solving a sequence of optimization problems from coarse to fine scale via estimation of the basis coefficients μ_{k}. This is analogous to multigrid methods, but here the notion of refinement from coarse to fine is accomplished by increasing the number of basis components d. As the number of basis functions increases, smaller and smaller variabilities between the template and target are accommodated. The basis coefficients are determined by gradient descent, i.e.,
also ∇ is a fixed step size and λ_{k }are the eigenvalues of the eigenvectors φ_{k}.
The computation of the fusion transformation (step 370) using the synthesis equation is presented in the flow chart of FIG. 8. Equation (12) is used to initialize the value of the displacement field u(x)=u^{(0)}(x) (step 800). The basis coefficients μ_{k}=μ_{k}^{(0) }are set equal to zero and the variables β_{i}, A, and b are set equal to the solution of equation (11) (step 802). Equation (13) is then used to estimate the new values of the basis coefficients μ_{k}^{(n+1) }given the current estimate of the displacement field u^{(n) }(x) (step 804). Equation (15) is then used to compute the new estimate of the displacement field u^{(n) }(x) given the current estimate of the basis coefficients μ_{k}^{(n) }(step 806). The next part of the computation is to decide whether or not to increase the number d of basis functions φ_{k }used to represent the transformation (step 808). Increasing the number of basis functions allows more deformation. Normally, the algorithm is started with a small number of basis functions corresponding to low frequency eigen functions and then on defined iterations the number of frequencies is increased by one (step 810). This coarsetofine strategy matches larger structures before smaller structures. The preceding computations (steps 804810) are repeated until the computation has converged or the maximum number of iterations is reached (step 812). The final displacement field is then used to transform the template image (step 814).
Once CPU 204 determines the transform from the synthesis equation fusing both landmark manifold information and image data, CPU 204 uses this transform to register the template image with the target image (step 380).
The spectrum of the second transformation, h, is highly concentrated around zero. This means that the spectrum mostly contains low frequency components. Using the sampling theorem, the transformation can be represented by a subsampled version provided that the sampling frequency is greater than the Nyquist frequency of the transformation. The computation may be accelerated by computing the transformation on a coarse grid and extending it to the full voxel lattice e.g., in the case of 3D images, by interpolation. The computational complexity of the algorithm is proportional to the dimension of the lattice on which the transformation is computed. Therefore, the computation acceleration equals the ratio of the full voxel lattice to the coarse computational lattice.
One of ordinary skill in the art will recognize that composing the landmark transformations followed by the elastic basis transformations, thereby applying the fusion methodology in sequence can provide an alternatively valid approach to hierarchial synthesis of landmark and image information in the segmentation.
Another way to increase the efficiency of the algorithm is to precompute the Green'"'"'s functions and eigen functions of the operator L and store these precomputed values in a lookup table. These tables replace the computation of these functions at each iteration with a table lookup. This approach exploits the symmetry of Green'"'"'s functions and eigen functions of the operator L so that very little computer memory is required. In the case of the Green'"'"'s functions, the radial symmetry is exploited by precomputing the Green'"'"'s function only along a radial direction.
The method described for fusing landmark information with the image data transformation can be extended from landmarks that are individual points (0dimensional manifolds) to manifolds of dimensions 1, 2 and 3 corresponding to curves (1dimensional), surfaces (2dimensional) and subvolumes (3dimensional).
For example, FIG. 4 shows a template image 400 of a section of a brain with 1dimensional manifolds 402 and 404 corresponding to target image 406 1dimensional manifolds 408 and 410 respectively. FIG. 5 shows a template image 500 of a section of a brain with 2dimensional manifold 502 corresponding to target image 504 2dimensional manifold 506. FIG. 6 shows a template image 600 of a section of a brain with 3dimensional manifold 602 corresponding to target image 604 3dimensional manifold 606.
As with the point landmarks, these higher dimensional manifolds condition the transformation, that is we assume that the vector field mapping the manifolds in the template to the data is given. Under this assumption the manuallyassisted deformation (step 350, FIG. 3) becomes the equalityconstrained Bayesian optimization problem:
If M(i) is a smooth manifold for i=0, 1, 2, 3, the solution to this minimization is unique satisfying L^{†}Lû(x)=0, for all template points in the selected manifold. This implies that the solution can be written in the form of a Fredholm integral equation:
and G the Green'"'"'s function of L.
When the manifold is a subvolume, M(3),dS is the Lebesgue measure on R^{3}. For 2dimensional surfaces, dS is the surface measure on M(2), For 1dimensional manifolds (curves), dS is the line measure on M(1) and for point landmarks, M(0), dS is the atomic measure. For point landmarks, the Fredholm integral equation degenerates into a summation given by equation (10).
When the manifold of interest is a smooth, 2dimensional surface, the solution satisfies the classical Dirichlet boundary value problem:
The Dirichlet problem is solved using the method of successive over relaxation as follows. If u^{k}(x) is the estimate of a deformation field at the k^{th }iteration, the estimate at the (k+1)^{th }iteration is given by the following update equation:
where α is the over relaxation factor.
It is also possible to compute the transform (step 370) with rapid convergence by solving a series of linear minimization problems where the solution to the series of linear problems converges to the solution of the nonlinear problem. This avoids needing to solve the nonlinear minimization problem directly. Using a conjugate gradient method, the computation converges faster than a direct solution of the synthesis equation because the basis coefficients μ_{k }are updated with optimal step sizes.
Using the conjugate gradient, the displacement field is assumed to have the form
Begin by assuming that f(x) is fixed. This is generalized below. The eigen functions in the expansion are all real and follow the assumption that {φ_{i}(x)} are R^{3 }valued.
The minimization problem is solved by computing
to update the basis coefficients in equation (21) where μ_{j}=0,j=0 . . . d initially, Δ_{j }is computed using the equation
where h_{i}(x)=∇T_{x−u(x)}·φ_{i}(x), and where θ_{ij}(x)=φ_{j}(x)·φ_{j}(x). The notation f·g is the innerproduct, i.e.,
Similarly, since u(x) is written in the series expansion given in equations (21) and (22), the identical formulation for updating β_{i }arises. Accordingly, the fusion achieved by the present invention results. Computation of equation (23) repeats until all Δ_{j }fall below a predetermined threshold solving for each Δ_{j }in sequence of increasing j, and Δ_{j }is computed using the values of Δ_{k }for 0≦k<j.
A further improvement over prior art image registration methods is achieved by computing the required transforms using fast Fourier transforms (FFT). Implementing an FFT based computation for image registration using a synthesis equation, as required at step 370 of FIG. 3, provides computational efficiency. However, to exploit the known computational efficiencies of FFT transforms, the solution of the synthesis equation must be recast to transform the inner products required by iterative algorithms into shift invariant convolutions.
To make the innerproducts required by the iterative algorithms into shift invariant convolution, differential and difference operators are defined on a periodic version of the unit cube and the discrete lattice cube. Thus, the operators are made cyclostationary, implying their eigen functions are always of the form of complex exponentials on these cubes having the value:
r=1, 2, 3 with x=(x_{1},x_{2},x_{3})ε[0, 1]^{3},
ω_{ki}=2πk_{i}, i=1, 2, 3, and the Fourier basis for periodic functions on [0, 1]^{3 }takes the form
This reformulation supports an efficient implementation of the image registration process using the FFT. Specifically, if step 370 of FIG. 3, computing the registration transform fusing landmark and image data, is implemented using the conjugate gradient method, the computation will involve a series of inner products. Using the FFT exploits the structure of the eigen functions and the computational efficiency of the FFT to compute these innerproducts.
For example, one form of a synthesis equation for executing Step 370 of FIG. 3 will include the following three terms:
Each of theses terms must be recast in a suitable form for FFT computation. One example of a proper reformulation for each of these terms is:
Term 1:
where c_{k}^{(r)}=[c_{1k}^{(r)}, c_{2k}^{(r)}, c_{3k}^{(r)}]^{t}. This equation is computed for all k by a Fourier transformation of the function.
and hence can be computed efficiently using a 3D FFT.
Term 2:
The integral in the above summation for all k can be computed by Fourier transforming the elements of the 3×3 matrix:
evaluated at ω_{k}+ω_{j. }Because this matrix has diagonal symmetry, the nine FFTs in this reformulation of term 2 can be computed efficiently using six three dimensional FFTs evaluated at ω_{k}+ω_{j. }
Term 3:
Using the exact form for the eigen functions we can rewrite the above equation as
This summation is precisely the inverse Fourier transforms of the functions
and hence can be computed efficiently by using a 3D FFT.
One of ordinary skill in the art will recognize that restructuring the computation of registration transforms using FFTs will improve the performance of any image registration method having terms similar to those resulting from a synthesis equation fusing landmark and image data. Improvements results from the fact that many computer platforms compute FFTs efficiently; accordingly, reformulating the registration process as an FFT computation, makes the required computations feasible.
A distance function used to measure the disparity between images is the Gaussian square error distance ∫T(x−u(x))−S(x)^{2}dx. There are many other forms of an appropriate distance measure. More generally, distance functions, such as the correlation distance, or the Kullback Liebler distance, can be written in the form
An efficient convolution implementation can be derived using the FFT for arbitrary distance functions. Computing the fusing transform using the image data follows the equation:
where D(.,.) is a distance function relating points in the template and target images. The displacement field is assumed to have the form:
is fixed. The basis coefficients {μ_{k}} are determined by gradient descent, i.e.,
where the gradient is computed using the chain rule and is given by the equation
where D′ (.,.) is the derivative with respect to the first argument. The most computationally intensive aspect of the algorithm is the computation of the term
Using the structure of the eigen functions and the computational efficiency of the FFT to compute these innerproducts, the above term can be written as
where
c_{k}^{(r)}=[c_{1k}^{(r)}, c_{2k}^{(r)}, c_{3k}^{(r)}]^{t}. This equation is computed for all k by a Fourier transformation of the function
and hence can be computed efficiently using a 3D FFT.
The following example illustrates the computational efficiencies achieved using FFTs for image registration instead of direct computation of innerproducts. Assuming that a target image is discretized on a lattice having N^{3 }points, each of the innerproducts in the algorithm, if computed directly, would have a computational complexity of the order (N^{3})^{2}. Because the innerproducts are computationally intensive, the overall complexity of image registration is also (N^{3})^{2}. In contrast, each of the FFTs proposed has a computational complexity on the order of N^{3}log_{2 }N^{3}. The speed up is given by the ratio N^{6}/(N^{3}log_{2 }N^{3})=N^{3}/(3 log_{2 }N). Thus the speed up is 64 times for a 16×16×16 volume and greater than 3.2×10^{4 }speed up for a 256×256×256 volume.
A further factor of two savings in computation time can be gained by exploiting the fact that all of the FFTs are real. Hence all of the FFTs can be computed with corresponding complex FFTs of half the number of points. For a development of the mathematics of FFTs see, A. V. Oppenheim and R. W. Schafer, Digital Signal Processing, PrenticeHall, New Jersey, 1975 (hereinafter referred to as Oppenheim).
Alternative embodiments of the registration method described can be achieved by changing the boundary conditions of the operator. In the disclosed embodiment, the minimization problem is formulated with cyclic boundary conditions. One of ordinary skill in the art will recognize that alternative boundary conditions, such as the Dirichlet, Neumann, or mixed Dirichlet and Neumann boundary conditions are also suitable. The following equation is used in an embodiment of the present invention using one set of mixed Dirichlet and Neumann boundary conditions:
where the notation (xx_{i}=k) means x is in the template image such that x_{i}=k. In this case, the eigen functions would be of the form:
Modifying boundary conditions requires modifying the butterflies of the FFT from complex exponentials to appropriate sines and cosines.
In FIG. 7, four images, template image 700, image 704, image 706, and target image 708, illustrate the sequence of registering a template image and a target image. Template image 700 has 0dimensional landmark manifolds 702. Applying the landmark manifold transform computed at step 350 in FIG. 3 to image 700 produces image 704. Applying a second transform computed using the synthesis equation combining landmark manifolds and image data to image 700 produces image 706. Image 706 is the final result of registering template image 700 with target image 708. Landmark manifold 710 in image 708 corresponds to landmark manifold 702 in template image 700.
Turning now to techniques to register images which may possess large deformation characteristics using diffeomorphisms, large deformation transform functions and transformations capable of matching images where the changes from one image to the other are greater than small, linear, or affine. Use of large deformation transforms provide for the automatic calculation of tangents, curvature, surface areas, and geometric properties of the imagery.
2. Methods for Large Deformation Landmark Based and Image Based Transformations
An embodiment consistent with the present invention maps sets of landmarks in imagery {x_{i}, i=1,2, . . . , N} ⊂ Ω into target landmarks {y_{i}, i=1, . . . , N}, and or imagery I_{0 }into target I_{1 }both with and without landmarks. For example when there is a welldefined distance function D(u(T)) expressing the distance between the landmarks and or imagery. The large deformation maps h: Ω→Ω are constructed by introducing the time variable,
The large deformation maps are constrained to be the solution h(x, T)=x−u(x,T) where u(x, T) is generated as the solution of the ordinary differential equation
assuming that the {φ_{k}} forms a complete orthonormal base.
A differomorphism for the landmark and image matching problem is given by the mapping of the imagery given by ĥ(x,T)=x−û(x,T) satisfying the Ordinary Differential Equation (O.D.E.)
L is preferably a linear differential operator with the φ_{k }forming a complete orthonormal base as the eigenfunctions Lφ_{k}=λ_{k}φ_{k}.
2.1 Large Deformation Landmark Matching
In order to register images with large deformation using a landmark matching technique, N landmarks are identified in the two anatomies targert {x_{i}, y_{i}, i=1,2, . . . , N}. The landmarks are identified with varying degrees of accuracy {Σ_{i}, i=1, . . . , N}, the Σ_{i}, 3×3 covariance matrices. The distance between the target and template imagery landmarks is preferably defined as
A preferable differomorphism is the minimizer of Eqns. 40, 41 with D(u(T)) the landmark distance:
A method for registering images consistent with the present invention preferably includes the following steps:
STEP 0: Define a set of landmarks in the template which can be easily identified in the target {x_{i}:x_{i }εΩ, i=1,2, . . . , N} with varying degrees of accuracy in the target as {yi, i=1, . . . , N} with associated error covariance {Σ_{i}, i=1, . . . , N}, and initialize for n=0, ν.=0.
STEP 1: Calculate velocity and deformation fields:
STEP 2: Solve via a sequence of optimization problems from coarse to fine scale via estimation of the basis coefficients {ν_{k}}, analogous to multigrid methods with the notion of refinement from coarse to fine accomplished by increasing the number of basis components. For each ν_{k},
STEP 3: Set n←n+1, and return to Step 1.
2.2 Large Deformation Image Matching
Turning now to the technique where a target and template image with large deformation are registered using the image matching technique. The distance between the target and template image is chosen in accordance with Eqn. 47. A diffeomorphic map is computed using an image transformation operator and image transformation boundary values relating the template image to the target image. Subsequently the template image is registered with the target image using the diffromorphic map.
Given two images I_{0}, I_{1}, choose the distance
The large deformation image distance driven map is constrained to be the solution ĥ(x,T)=x−û(x,T) where
A method for registering images consistent with the present invention preferably includes the following steps:
STEP 0: Measure two images I_{0}, I_{1 }defined on the unit cube Ω≈[0,1]^{3}⊂R^{8}, initialize parameters for n=0, ν_{k}^{(n)}=0, k=0,1, . . . , and define distance measure D(u(T)) Eqn. 47.
STEP 1: Calculate velocity and deformation fields from ν^{(n)}:
STEP 2: Solve optimization via sequence of optimization problems from coarse to fine scale via reestimation of the basis coefficients {ν_{k}}, analogous to multigrid methods with the notion of refinement from coarse to fine accomplished by increasing the number of basis components. For each ν_{k},
STEP 3: Set n←n+1, and return to Step 1.
The velocity field can be constructed with various boundary conditions, for example ν(x,t)=0, xεδΩ and tε[0, T], u(x, 0)=ν(x, 0)=0. The differential operator L can be chosen to be any in a class of linear differential operators; we have used operators of the form (−aΔ−b∇∇·cI)^{p}, p≧1. The operators are 3×3 matrices
2.3 Small Deformation Solution
The large deformation computer algorithms can be related to the small deformation approach described in the '"'"'212 patent, by choosing ν=u, so that T=δ small, then approximate I−∇u(·,σ)≈I for σε[0, δ), then û(x, δ)=σ(x)δ and defining u(x)≈u(x, δ), so that Eqn. 55 reduces to the small deformation problems described in the '"'"'212 patent:
3. Composing Large Deformation Transformation Unifying Landmark and Image Matching
The approach for generating a hierarchical transformation combining information is to compose the large deformation transformations which are diffeomorphisms and can therefore be composed, h=h_{n}^{o}. . . h_{2 }o_{h}_{1}. Various combinations of transformations may be chosen, including the affine motins, rigid motions generated from subgroups of the generalized linear group, large deformation landmark transformations which are diffeomorphisms, or the high dimensional large deformation image matching transformation (the dimension of the transformations of the vector fields are listed in increasing order). Since these are all diffeomorphisms, they can be composed.
4. Fast Method for Landmark Deformations Given Small Numbers of Landmarks
For small numbers of landmarks, we reparameterize the problem using the Lagrangian frame, discretizing the optimization over space time Ω×T, into N functions of time T. This reduces the complexity by an order of magnitude given by the imaging lattice [Ω].
For this, define the Lagrangian positions of the Nlandmarks x_{i}, i=1, . . . , N as they flow through time φ_{i}(·), i=1, . . . , N, with the associated 3Nvector
The particle flows Φ(t) are defined by the velocities ν(·) according to the fundamental O.D.E.
It is helpful to define the 3N×3N covariance matrix K(Φ(t)):
The inverse K(Φ(t))^{−1 }is an N×N matrix with 3×3 block entries (K(Φ(t))^{−1})_{ij}, i, j=1, . . . , N.
For the landmark matching problem, we are given N landmarks identified in the two anatomies target {x_{i}, y_{i}, i=1,2, . . . , N}, identified with varying degrees of accuracy {Σ_{i.}, i=1, . . . N}, the Σ_{i}, 3×3 covariance matrices.
The squared error distance between the target and template imagery defined in the Lagrangian trajectories of the landmarks becomes
Then a preferable diffeomorphism is the minimizer of Eqns. 40, 41 with D_{1}(Φ(T)) the landmark distance:
A fast method for small numbers of Landmark matching points exploits the fact that when there are far fewer landmarks N than points in the image x εΩ, there is the following equivalent optimization problem in the NLagrangian velocity fields φ(x_{i}, ·), I=1, . . . , N which is more computationally efficient than the optimization of the Eulerian velocity ν(x_{i}, ·), x εΩ, (Ω>>N). Then, the equivalent optimization problem becomes
This reduces to a finite dimensional problem by defining the flows on the finite grid of times, assuming stepsize δ with Lagrangian velocities piecewise constant within the quantized time intervals:
Then the finite dimensional minimization problem becomes
The Method:
In order to properly register the template image with the target image when a small number of landmarks have been identified, the method of the present embodiment utilizes the Largrangian positions. The method for registering images consistent with the present invention includes the following steps:
STEP 0: Define a set of landmarks in the template which can be easily identified in the target {x_{i}: x_{i }εΩ, i=1,2, . . . , N}, with varying degrees of accuracy in the target as {y_{i}, i=1, . . . , N} with associated error covariance {Σ_{i}, i=1, . . . , N}, and initialize for n=0, φ^{(n)}(x_{i}, ·)=x_{i}.
STEP 1: Calculate velocity and deformation fields:
STEP 2: Solve via estimation of the Lagrangian positions φ(k), k=1, . . . , K. For each φ(x_{i}, k),
STEP 3: Set n←n+1, and return to Step 1.
STEP 4: After stopping, then compute the optimal velocity field using equation 62 and transform using equation 64.
5. Fast Greedy Implementation of large Deformation Image Matching
For the image matching, discretize space time Ω×T into a sequence of indexed in time optimizations, solving for the locally optimal at each time transformation and then forward integrate the solution. This reduces the dimension of the optimization and allows for the use of Fourier transforms.
The transformation h(;, t):Ω→Ω where h(x, t)=x−u(x, t), and the transformation and velocity fields are related via the O.D.E.
Preferably the deformation fields are generated from the velocity fields assumed to be piecewise constant over quantized time increments,
the quantized time increment. Then the sequence of deformations u(x,t_{i}), i=1, . . . I is given by
For δ small, approximate ∇u(x,σ;)≈∇u(x,t_{i}),σε[t_{i},t_{i+1}], then the global optimization is solved via a sequence of locally optimal solutions according to for t_{i}, I=1, . . . , I,
The sequence of locally optimal velocity fields {circumflex over (ν)}(x,t_{i}),i=1, . . . I satisfy the O.D.E.
Examples of boundary conditions include ν(x,t)=0, xεδΩ and tε[0,T]and L the linear differential operator L=−a∇^{2}−b∇·∇+cI. The body force b(x−u(x,t)) is given by the variation of the distance D(u) with respect to the field at time t. The PDE is solved numerically (G. E. Christensen, R. D. Rabbitt, and M. I. Miller, “Deformable templates using large deformation kinematics,” IEEE Transactions on Image Processing, 5(10):14351447, October 1996 for details (hereinafter referred to as Christensen).
To solve for fields u(·, t_{i}) at each t_{i}, expand the velocity fields Σ^{k}V_{k}(t_{i})φ_{k}(·). ν(·,t_{i})=Σ^{k }Vk(t_{i})φk(·). To make the innerproducts required by the iterative algorithms into shift invariant convolution, force the differential and difference operators L to be defined on a periodic version of the unit cube Ω≈[0,1]^{3 }and the discrete lattice cube {0,1, . . . , N−1}^{3}. f·g denotes the innerproduct
The operators are cyclostationary in space, implying their eignen functions are of the form of complex exponentials on these cubes:
and the Fourier basis for periodic functions on [0,1]^{3 }takes the form e^{j<ω}^{k,}^{x>}, <ω_{k}, x>=ω_{k1}x_{1}+ω_{k2}x_{2}+ω_{k3}x_{3}. On the discrete N^{3}=periodic lattice,
This supports an efficient implementation of the above algorithm exploiting the Fast Fourier Transform (FFT).
Suppressing the d subscript on ν_{k,d }and the summation from d=1 to 3 for the rest of this section simplifies notation. The complex coefficients ν_{k}(t_{i})=a_{k}(t_{i})+b_{k}(t_{i}) have complexconjugate symmetry because they are computed by taking the FFT of a real valued function as seen later in this section. In addition, the eigen functions have complexconjugate symmetry due to the 2π periodicity of the complex exponentials. Using these two facts, it can be shown that the vector field
is R^{3 }valued even though body ν_{k}(t_{i}) and φ_{k}(x) are complex valued. The minimization problem is solved by computing for each v_{k}=a_{k}+b_{k},
and g_{k }ε{a_{k}, b_{k}}. Combining Eqs. 70 and 71 and taking derivatives gives
Consider the computation of the following terms of the algorithm from equations 73,
Computation of Term 1:
The first term given by
can be written as
This equation can be computed efficiently using three 3D FFTs of the form
and s=1,2,3. These FFTs are used to evaluate Eq. 72 by noticing:
Computation of Term 2:
The second term given by
can be computed efficiently using three 3D FFTs. Specifically the 3D FFTs are
for s=1,2,3 and
Using the FFT to compute the terms in the method provides a substantial decrease in computation time over brute force computation. For example, suppose that one wanted to process 256^{3 }voxel data volumes. The number of iterations is the same in both the FFT and brute force computation methods and therefore does not contribute to our present calculation. For each 3D summation in the method, the brute force computation requires on the order N^{6 }computations while the FFT requires on the order 3N^{3}log_{2}(N) computations. For N^{3}=256^{3 }voxel data volumes this provides approximately a 7×10^{5 }speed up for the FFT algorithm compared to brute force calculation.
6. Rapid Convergence Algorithm for Large Deformation Volume Transformation
Faster converging algorithms than gradient descent exist such as the conjugate gradient method for which the basis coefficients ν_{k }are updated with optimal step sizes.
The identical approach using FFTs follows as in the '"'"'212 patent. Identical speedups can be accomplished; see the '"'"'212 patent.
6.1 An Extension to General Distance Functions
Thus far only the Gaussian distance function has been described measuring the disparity between imagery has been described as ∫I_{0}(x−u(x))−I_{1}(x)^{2}dx. More general distance functions can be written as ∫D(I_{0}(x−u(x))), I_{1}(x))dx. A wide variety of distance functions are useful for the present invention such as the correlation distance, or the Kullback Liebler distance can be written in this form.
6.2 Computational Complexity
The computational complexity of the methods herein described is reduced compared to direct computation of the innerproducts. Assuming that the image is discretized on a lattice of size N^{3 }each of the innerproducts in the algorithm, if computed directly, would have a computational complexity of O((N^{3})^{2}). As the innerproducts are most computationally intensive, the overall complexity of the method is O((N^{3})^{2}). Now in contrast, each of the FFTs proposed have a computational complexity of O(N^{3 }log_{2}N^{3}), and hence the total complexity of the proposed algorithm is O(N^{3 }log_{2 }N^{3}). The speed up is given by the ratio N^{6 }/(N^{3 }log2 N^{3})=N^{3}/(3 log_{2 }N). Thus the speed up is 64 times for 16×16×16 volume and greater than 3.2×10^{4 }speed up for a 256×256×256 volume.
A further factor of two savings in computation time can be gained by exploiting the fact that all of the FFTs are real. Hence all of the FFTs can be computed with corresponding complex FFTs of half the number of points (see Oppenheim).
6.3 Boundary Conditions of the Operator
Additional methods similar to the one just described can be synthesized by changing the boundary conditions of the operator. In the previous section, the inimization problem was formulated with cyclic boundary conditions. Alternatively, we could use the mixed Dirichlet and Neumann boundary conditions corresponding to
for i,j=1,2,3; i≠j; k=0,1 where the notation (xx_{i}=k) means xεΩ such that x_{i}=k. In this case the eigen functions would be of the form
The implementation presented in Section 5 is modified for different boundary conditions by modifying the butterflys of the FFT from complex exponentials to appropriate sines and cosines.
7.0 Large Deformation Diffeomorphisms on a Sphere
In many applications, image registration algorithms operate on images of spherical objects. Accordingly, the following section extends the diffeomorphic landmark matching technique introduced above to create an embodiment of the present invention for registering images of objects having spherical geometries. A method for registering images using large deformation diffeomorphisms on a sphere comprises selecting a coordinate frame suitable for spherical geometries and registering the target and template image using a large deformation diffeomorphic transform in the selected coordinate frame.
While there are many applications that benefit from an image registration technique adapted to spherical geometries, one such example, registering brain images, is discussed herein to illustrate the technique. One skilled in the are will recognize that other imaging applications are equally suited for this technique, such as registering images of other anatomical regions and registering nonanatomical imagery containing spherical regions of interest.
An application of brain image registration is the visualization of cortical studies. Current methods of visualizing cortical brain studies use flat maps. Although flat maps bring the buried cortex into full view and provide compact representations, limitations are introduced by the artificial cuts needed to preserve topological relationships across the cortical surface. Mapping the cortical hemisurface to a sphere, however, allows points on the surface to be represented by a twodimensional coordinate system that preserves the topology. Spherical maps allow visualization of the full extent of sulci and the buried cortex within the folds. An embodiment consistent with the present invention generates large deformation diffeomorphisms on the sphere S^{2 }which has a onetoone correspondence with a reconstructed cortical surface.
These diffeomorphisms are generated as solutions to the transport equation:
For the spherical coordinate maps, the final transformed coordinate map is defined as φ(·,1)ε S^{2 }and controlled through the ODE in equation 76 by the velocity field v(·, t), tε [0,1] constrained to be in the tangent space of the sphere.
The template image and target image spheres are characterized by the set of landmarks {x_{n}, y_{n}, n=1,2, . . . N}εS^{2}. Diffeomorphic matches are constructed by forcing the velocity fields to be associated with quadratic energetics on S^{2}×[0,1]. The diffeomorphic landmark matching is constructed to minimize a running smoothness energy on the velocity field as well as the end point distance between the template and target landmarks.
Given noisy template landmarks x_{n}εS^{2 }matched to target landmarks y_{n }εS^{2 }measured with error variances σ_{n}^{2}, a suitable diffeomorphism is given by:
where L is a differential operator, Ψ(x,y) is the solid angle between points x,y on the sphere, E_{i}, i=1,2 are the coordinate frames on the sphere and dμ(x)=sinΨdΨdθ is the surface measure.
The velocity field should be represented so that
are in the span of E_{1 }and E_{2}, a basis that spans the tangent space of the sphere S^{2}. An azimuth/elevation parameterization for this space is: θε[0, 2π], ψε(0, π), let x be the chart:
The coordinate frames are given by
according to:
However, these frames vanish at ψ=0, π, namely the north and south poles of the sphere and leave these two points transformationinvariant. When no nonvanishing continuous vector field exists on the sphere S^{2}, an alternative parameterization that has nonzero coordinate frames everywhere except at one point is appropriate. This is acceptable because each hemisphere of the brain is mapped to a sphere individually, and the area they are attached to each other is transformation invariant. For some applications, this conforms to an anatomical constraint, e.g., the point at which the brain attached to the spine. Moreover, the location producing zerovalued coordinate frames can be user selected by computing a rigid alignment to move this location in the image. One example of such an alignment involves aligning the sets of landmarks by computing a rigid transform (which, in a spherical coordinate frame, is a rotation) and then applying another rigid transform to ensure that the fixed points are aligned correctly.
An embodiment of the present invention uses stereographic projection to parameterize the nit sphere corresponding to the right hemisphere of the brain, with the center shifted to (−1,0,0) and take the shadow (u,v) of each point in the yz plane while shining a light from point (−2,0,0). Note that the shadow of the point from where the light source is located is a infinity, and this becomes the point where the coordinate frames vanish. The location of the light source and the projection plane can be adjusted to place the transformation invariant point as needed. Let P denote the stereographic projection from S^{2}\(−2,0,0) to (u,v)εR^{2}.
and the chart P^{−1 }mapping (u,v) back into S^{2 }is given by:
The coordinate frames on the sphere E_{1}(θ,ψ), E_{2}(θ,ψ) become:
Both E_{1 }and E_{2 }are 0 when p=(−2,0,0), but that is the transformation invariant point on the sphere representing the right hemisphere.
A similar parameterization can be obtained for the left hemisphere by shifting the center to (1,0,0) and shining the light from (2,0,0) which becomes the transformation invariant point. Accordingly, an appropriate coordinate frame for images containing objects having spherical geometry is generated for subsequent image registration.
Large deformation landmark matching on the sphere proceeds with diffeomorphisms constructed as solutions to the transport equation by forcing velocity fields to minimize quadratic energetics on S^{2}×[0,1] defined by the Laplacian. The Laplacian operator on functions defined on the sphere in azimuthelevation parameters is:
The energy associated with
is:
The covariance operator is then computed using spherical harmonics. The covariance operator (which is the Green'"'"'s function squared of the Laplacian operator) will determine the solution to the diffeomorphic matching (see equation 93 below). Spherical harmonics Y_{nm }of order n form a complete orthogonal basis of the sphere that correspond to the eigenfunctions of the differential operator ∇^{2}Y_{nm}=−n(n+1)Y_{nm}. There are (2 n+1) spherical harmonics of order n for each n and they are of the even and odd harmonic form with 0≦m≦n:
with P_{n}^{m }the associated Legendre polynomials, and k_{nm}
normalization constants.
The Green'"'"'s function on the sphere for two points p and q becomes
Orthonormality of spherical harmonics and the addition theorem for spherical harmonics result in a covariance operator of the form:
where Ψ(p,q)=arccos(p_{1}q_{1}+p_{2}q_{2}+p_{3}q_{3}) is the solid angle between the points p and q. The covariance is shift invariant on the sphere and coordinatefree.
The resulting 2×2 covariance matrix K(p,q) and the 2N×2N matrix K(φ(t)) are:
Instead of trying to solve the optimization problem directly in the velocity fields over all of spacetime ν(x,t)εS^{2}×[0, 1], the Nlandmark trajectories and velocities are estimated as follows.
The large deformation landmark matching transformation on the sphere is given by the diffeomorphism:
with {circumflex over (ν)}solving the minimum problem
where dμ(x)=sinφdφdθ and {circumflex over (ν)}(x,t) satisfies
where Ψ(y_{n}, φ(x_{n}, 1)) is the solid angle between the target landmark y_{n }and the final position of the template landmark φ(x_{n}, 1).
The algorithm for landmark matching reduces the problem to a fine dimensional problem by defining the flows on the finite grid of fixed times of size δ, t_{k}=kδ,
An embodiment of the present invention uses a gradient algorithm to register the images using the selected spherical coordinate frame and the derived large deformation diffeomorphic transform.
The finite dimensional minimization becomes {circumflex over (ν)}_{t}(x_{n}, 0)=0, n=1, . . . , N, i=1,2 and
The gradient algorithm for minimizing Eqn. 95 initializes with m=0 and ν^{(O)}(x_{n}, t_{k})=0, n=1, . . . N, k=1, . . . , K, and iterates for m=0,1, . . .
Calculate gradient perturbation for each ν_{1}(x_{n}, t_{k}), n=1, N, k=1, . . . , K, i=1,2:
δ[t_{k}−1]=1 for t_{k}=1, and 0 otherwise, Δ is the gradient step.
After stopping, define the final iterate as {circumflex over (ν)}_{i}=ν_{i}^{(m+1)}, and for all x εS^{2 }
8. Apparatus for Image Registration
FIG. 2 shows an apparatus to carry out an embodiment of this invention. A medial imaging scanner 214 obtains image 100 and 120 and stores them in computer memory 206 which is connected to computer control processing unit (CPU) 204. One of the ordinary skill in the art will recognize that a parallel computer platform having multiple CPUs is also a suitable hardware platform for the present invention, including, but not limited to, massively parallel machines and workstations with multiple processors. Computer memory 206 can be directly connected to CPU 204, or this memory can be remotely connected through a communications network.
The methods described herein use information either provided by an operator, stored as defaults, or determined automatically about the various substructures of the template and the target, and varying degrees of knowledge about these substructures derived from anatomical imagery, acquired from modalities like CT, MRI, functional MRI, PET, ultrasound, SPECT, MEG, EEG or cryosection. For example, an operator can guide cursor 210 using pointing device 208 to select in image 100.
The foregoing description of the preferred embodiments of the present invention has been provided for the purpose of illustration and description. It is not intended to be exhaustive or to limit the invention to the precise forms disclosed. Obviously many modifications, variations and simple derivations will be apparent to practitioners skilled in the art. The embodiments were chosen and described in order to best explain the principles of the invention and its practical application, thereby enabling others skilled in the art to be understand the invention for various embodiments and with various modifications are suited to the particular use contemplated. It is intended that the scope of the invention be defined by the following claims and their equivalents.