System for moving first insertable instrument and second insertable instrument, controller, and computer-readable storage device

• US 10,638,915 B2
• Filed: 05/10/2018
• Issued: 05/05/2020
• Est. Priority Date: 02/10/2016
• Status: Active Grant

• Current Assignee: Olympus Corporation
• Original Assignee: Olympus Corporation
• Inventors: Nishizawa, Koji
• Primary Examiner(s): Hilaire, Clifford
• Application Number:
  US15/976,142
• Publication Number:
  US 20180256008A1
• Time in Patent Office: Days
• CPC Class Codes: A61B 1/00006 of control signals
• CPC Class Codes: A61B 1/00149 using articulated arms
• CPC Class Codes: A61B 1/05 characterised by the image sensor, e.g. camera, being in the distal end portion
• CPC Class Codes: A61B 2034/2055 Optical tracking systems
• CPC Class Codes: A61B 2034/2072 Reference field transducer attached to an instrument or patient
• CPC Class Codes: A61B 34/20 Surgical navigation systems; Devices for tracking or guiding surgical instruments, e.g. for frameless stereotaxis
• CPC Class Codes: G06T 2207/10068 Endoscopic image
• CPC Class Codes: G06T 2207/30204 Marker
• CPC Class Codes: G06T 2207/30244 Camera pose
• CPC Class Codes: G06T 7/73 using feature-based methods
• CPC Class Codes: H04N 23/555 for picking-up images in sites, inaccessible due to their dimensions or hazardous conditions, e.g. endoscopes or borescopes
• CPC Class Codes: H04N 23/66 Remote control of cameras or camera parts, e.g. by remote control devices

Abstract

A medical system having a first insertable instrument having a first image sensor; a second insertable instrument having a second image sensor; a first arm configured to move the first insertable instrument; and a controller configured to: acquire a position and orientation of the second insertable instrument in a predetermined coordinate system; calculate a position and direction of an optical axis of the second image sensor based on the position and orientation of the second insertable instrument; acquire a position and orientation of the first insertable instrument in the predetermined coordinate system; calculate a first operation amount of the first arm to move the first image sensor such that the optical axis of the second image sensor substantially coincides with an optical axis of the first image sensor, based on the position and orientation of the first insertable instrument; and control the first arm based on the first operation amount.

Claims

1. A medical system comprising:

a first insertable instrument comprising a first image sensor;

a second insertable instrument comprising a second image sensor;

an arm configured to support and move the first insertable instrument; and

a controller configured to;

acquire a position and orientation of the second insertable instrument in a predetermined coordinate system;

calculate a position and direction of an optical axis of the second image sensor based on the position and orientation of the second insertable instrument;

acquire a position and orientation of the first insertable instrument in the predetermined coordinate system;

calculate a first operation amount of the arm to move the first image sensor such that the optical axis of the second image sensor substantially coincides with an optical axis of the first image sensor to make a field-of-view center of the first image sensor and a field-of-view center of the second image sensor coincide with each other, based on the position and orientation of the first insertable instrument;

control the arm, based on the first operation amount of the arm, to move the first insertable instrument; and

after controlling the arm, based on the first operation amount of the arm, to move the first insertable instrument, control the arm to move the first insertable instrument such that the first insertable instrument is withdrawn from the field-of-view center of the second image sensor.

2. The medical system according to claim 1,

wherein the controller is configured to store a position and orientation of the first insertable instrument where the optical axis of the second image sensor coincides with the optical axis of the first image sensor.

3. The medical system according to claim 2,

wherein the controller is configured to, after controlling the arm to move the first insertable instrument such that the first insertable instrument is withdrawn from the field-of-view center of the second image sensor, control the arm to move the first insertable instrument to the stored position and orientation such that the optical axis of the second image sensor substantially coincides with the optical axis of the first image sensor.

4. A controller for controlling a medical system comprising:

a first insertable instrument comprising a first image sensor;

a second insertable instrument comprising a second image sensor; and

an arm configured to support and move the first insertable instrument,

wherein the controller comprises one or more processors configured to;

acquire a position and orientation of the second insertable instrument in a predetermined coordinate system;

calculate a position and direction of an optical axis of the second image sensor based on the position and orientation of the second insertable instrument;

acquire a position and orientation of the first insertable instrument in the predetermined coordinate system;

calculate a first operation amount of the arm to move the first image sensor such that the optical axis of the second image sensor substantially coincides with an optical axis of the first image sensor to make a field-of-view center of the first image sensor and a field-of-view center of the second image sensor coincide with each other, based on the position and orientation of the first insertable instrument;

control the arm, based on the first operation amount of the arm, to move the first insertable instrument; and

after controlling the arm, based on the first operation amount of the arm, to move the first insertable instrument, control the arm to move the first insertable instrument such that the first insertable instrument is withdrawn from the field-of-view center of the second image sensor.

5. The controller according to claim 4,

wherein the one or more processors are configured to store a position and orientation of the first insertable instrument where the optical axis of the second image sensor coincides with the optical axis of the first image sensor.

6. The controller according to claim 5,

wherein the one or more processors are configured to, after controlling the arm to move the first insertable instrument such that the first insertable instrument is withdrawn from the field-of-view center of the second image sensor, control the arm to move the first insertable instrument to the stored position and orientation such that the optical axis of the second image sensor substantially coincides with the optical axis of the first image sensor.

7. A computer readable storage device storing instructions for controlling a medical system comprising:

a first insertable instrument comprising a first image sensor;

a second insertable instrument comprising a second image sensor; and

an arm configured to support and move the first insertable instrument,

wherein the instructions cause a computer to perform processes comprising;

acquiring a position and orientation of the second insertable instrument in a predetermined coordinate system;

calculating a position and direction of an optical axis of the second image sensor based on the position and orientation of the second insertable instrument;

acquiring a position and orientation of the first insertable instrument in the predetermined coordinate system;

calculating a first operation amount of the arm to move the first image sensor such that the optical axis of the second image sensor substantially coincides with an optical axis of the first image sensor to make a field-of-view center of the first image sensor and a field-of-view center of the second image sensor coincide with each other, based on the position and orientation of the first insertable instrument;

controlling the arm, based on the first operation amount of the arm, to move the first insertable instrument; and

after controlling the arm, based on the first operation amount of the arm, to move the first insertable instrument, controlling the arm to move the first insertable instrument such that the first insertable instrument is withdrawn from the field-of-view center of the second image sensor.

8. The computer readable storage device according to claim 7,

wherein the instructions cause the computer to further perform processes comprising storing a position and orientation of the first insertable instrument where the optical axis of the second image sensor coincides with the optical axis of the first image sensor.

9. The computer readable storage device according to claim 8,

wherein the instructions cause the computer to further perform processes comprising, after controlling the arm to move the first insertable instrument such that the first insertable instrument is withdrawn from the field-of-view center of the second image sensor, controlling the arm to move the first insertable instrument to the stored position and orientation such that the optical axis of the second image sensor substantially coincides with the optical axis of the first image sensor.