Rotation compensation and image stabilization system

• US 7,783,133 B2
• Filed: 12/28/2006
• Issued: 08/24/2010
• Est. Priority Date: 12/28/2006
• Status: Active Grant

• Current Assignee: Ethicon Endo-Surgery, Inc. (Johnson & Johnson)
• Original Assignee: Ethicon Endo-Surgery, Inc. (Johnson & Johnson), Microvision, Inc.
• Inventors: Metting, Frank B. III, Luanava, Selso, Dunki-Jacobs, Robert J.
• Primary Examiner(s): Bella; Matthew C
• Assistant Examiner(s): Rush; Eric
• Application Number:
  US11/648,857
• Publication Number:
  US 20080159653A1
• Time in Patent Office: 1,335 Days
• Field of Search: 382/100, 382/128, 382/293, 382/296, 600/101, 600/103, 600/109, 600/110, 600/112, 600/117, 600/118, 600/138, 600/139, 600/160, 600/173
• US Class Current: 382/296
• CPC Class Codes: A61B 1/04 combined with photographic or television appliances
• CPC Class Codes: A61B 1/3132 for laparoscopy
• CPC Class Codes: A61B 2034/2048 using an accelerometer or inertia sensor
• CPC Class Codes: A61B 2090/365 augmented reality, i.e. correlating a live optical image with another image
• CPC Class Codes: A61B 34/20 Surgical navigation systems; Devices for tracking or guiding surgical instruments, e.g. for frameless stereotaxis
• CPC Class Codes: A61B 5/067 using accelerometers or gyroscopes
• CPC Class Codes: A61B 90/361 Image-producing devices, e.g. surgical cameras

Abstract

An image processing system is arranged to provide rotation compensation and image stabilization in a video scope system such as in endoscopy and laparoscopy systems. The image processing functions may operate at real-time frame rates so that the resulting processed image is observable with no time lag. A rotation sensor is included in the system to sense the position of scope. The sensed scope rotation may be used to cause rotation of the collected image and/or an image displayed on a video monitor. The sensed rotation may be used to identify or calculate a coordinate transformation matrix that is used for processing the image data. The system may include a horizon lock mechanism that can be user-actuated to engage rotation compensation. When the horizon lock mechanism is not engaged, the output image is locked to the scope tip and rotates with rotation of the scope.

Claims

1. An image processing system for stabilizing and compensating image rotations, the image processing system comprising:

a console that is arranged to provide a processed image by applying a rotation matrix to image data in real-time, wherein the console is arranged to select the rotation matrix in response to positional information, wherein the console is further arranged to identify a singularity associated with the positional information such that the rotation matrix will not be altered when the singularity is identified;

a scope that comprises a tubular body with a distal tip;

a control body that is arranged in cooperation with the scope for guiding the scope during operation;

a video capture device that is located within one of the control body, the tubular body and the distal tip of the scope, wherein the video capture device is arranged to provide the image data associated with a field of view; and

a rotation sensor that is located within at least one of the control body and the scope, wherein the rotation sensor is arranged to determine the positional information and communicate the positional information to the console.

2. A method for stabilizing and compensating image rotations in an image processing system, the method comprising:

acquiring positional information from a rotation sensor, wherein the positional information is associated with a rotational position of a scope in the image processing system, and wherein acquiring positional information from the rotation sensor comprises;

acquiring positional information from a gyroscope that is located within a body of the scope such that the orientation of the body of the scope can be determined from the acquired positional information;

acquiring image data from the scope;

determining an orientation of the scope from the acquired positional information, wherein the orientation is determined relative to an artificial horizon;

selecting a rotation matrix from the determined orientation, wherein selecting the rotation matrix from the determined orientation comprises;

selecting one of a multiplicity of scan matrices based on the determined orientation; and

applying the rotation matrix to the acquired image data to provide processed image data, wherein the processed image data does not rotate relative to the artificial horizon.

3. The method of claim 2, wherein acquiring image data comprises at least one of:

assigning pixels from the image data to appropriate 2D locations in a scan image based on the selected one of the multiplicity of scan matrices, and capturing the 2D scan image data.

4. A method for stabilizing and compensating image rotations in an image processing system, the method comprising:

acquiring positional information from a rotation sensor, wherein the positional information is associated with a rotational position of a scope in the image processing system, and wherein acquiring positional information from the rotation sensor comprises;

acquiring positional information from a gyroscope that is located within a body of the scope such that the orientation of the body of the scope can be determined from the acquired positional information;

acquiring image data from the scope;

determining an orientation of the scope from the acquired positional information, wherein the orientation is determined relative to an artificial horizon;

selecting a rotation matrix from the determined orientation; and

applying the rotation matrix to the acquired image data to provide processed image data, wherein the processed image data does not rotate relative to the artificial horizon;

identifying a singularity associated with the acquired positional information, and maintaining the rotation matrix when the singularity is identified.

5. The method of claim 4, wherein identifying the singularity comprises:

evaluating an angle associated with the acquired positional information, and determining if the evaluated angle is within a range of values that is associated with an axis parallel to gravity.

6. An image processing system for stabilizing and compensating image rotations, the image processing system comprising:

a console that includes a real-time transformation block that is arranged to apply assign each pixel from a 1D image data stream to a location in a 2D scan image according to a scan assignment matrix, and wherein the real-time transformation block is arranged such that the 2D scan image is de-rotated in response to sensed positional information, wherein the console is further arranged to select the scan assignment matrix by selecting one of a multiplicity of previously calculated scan matrices in response to the sensed positional information;

a scope that comprises a body with a distal tip;

a video capture device that is located within the body of the scope, wherein the video capture device is arranged to provide the 1D image data stream from an image that is observed in a field of view;

a control body that is arranged in cooperation with the scope for guiding the scope during operation, wherein the control body is coupled to the console; and

a rotation sensor that is located within at least one of the control body and the body of the scope, wherein the rotation sensor is arranged to sense the positional information and communicate the sensed positional information to the console.