MOBILE MANEUVERABLE DEVICE FOR WORKING ON OR OBSERVING A BODY

• US 20150223725A1
• Filed: 06/28/2013
• Published: 08/13/2015
• Est. Priority Date: 06/29/2012
• Status: Abandoned Application

• Current Assignee: Charite-Universitatsmedizin Berlin, Fraunhofer Gesellschaft Zur Foerderung Der Angewandten Forsching E.V.
• Original Assignee: Charite-Universitatsmedizin Berlin, Fraunhofer Gesellschaft Zur Foerderung Der Angewandten Forsching E.V.
• Inventors: Keeve, Erwin, Uhlmann, Eckart, Engel, Sebastian, Winne, Christian
• Application Number:
  US14/411,602
• Publication Number:
  US 20150223725A1
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: A61B 1/00006 of control signals
• CPC Class Codes: A61B 1/00011 characterised by signal transmission
• CPC Class Codes: A61B 1/00147 Holding or positioning arrangements
• CPC Class Codes: A61B 1/00194 adapted for three-dimensional imaging
• CPC Class Codes: A61B 1/05 characterised by the image sensor, e.g. camera, being in the distal end portion
• CPC Class Codes: A61B 17/00234 for minimally invasive surgery A61B17/0218, A61B17/0469, A61B17/12013, A61B17/1285, A61B17/29, A61B17/320016 take precedence
• CPC Class Codes: A61B 2034/2048 using an accelerometer or inertia sensor
• CPC Class Codes: A61B 2034/2061 using shape-sensors, e.g. fiber shape sensors with Bragg gratings
• CPC Class Codes: A61B 2034/2065 Tracking using image or pattern recognition
• CPC Class Codes: A61B 2090/363 Use of fiducial points
• CPC Class Codes: A61B 2090/374 NMR or MRI
• CPC Class Codes: A61B 2090/3762 using computed tomography systems [CT]
• CPC Class Codes: A61B 2090/3954 magnetic, e.g. NMR or MRI
• CPC Class Codes: A61B 2090/3966 Radiopaque markers visible in an X-ray image
• CPC Class Codes: A61B 2560/0223 of calibration, e.g. protocols for calibrating sensors
• CPC Class Codes: A61B 34/20 Surgical navigation systems; Devices for tracking or guiding surgical instruments, e.g. for frameless stereotaxis
• CPC Class Codes: A61B 34/30 Surgical robots
• CPC Class Codes: A61B 5/065 Determining position of the probe employing exclusively positioning means located on or in the probe, e.g. using position sensors arranged on the probe
• CPC Class Codes: A61B 5/066 Superposing sensor position on an image of the patient, e.g. obtained by ultrasound or x-ray imaging
• CPC Class Codes: A61B 5/725 using specific filters therefor, e.g. Kalman or adaptive filters A61B5/316 take precedence
• CPC Class Codes: A61B 90/39 : Markers, e.g. radio-opaque or breast lesions markers
• CPC Class Codes: G06T 7/70 : Determining position or orientation of objects or cameras camera calibration G06T7/80

Abstract

The invention relates to a mobile, maneuverable device (1000) having a mobile device head (100), particularly a medical, mobile device head (100) having a distal end for the purpose of arrangement relative to a body, particularly insertion or attachment on the body, having at least one mobile device head (100) designed for the purpose of manual or automatic guidance, having a guide device (400) designed for the purpose of navigation, having an image data processing device (430) which compiles a map (470) of the environment by means of the image data, and having a navigation device which can indicate at least one position (480) of the device head (100) using the map, by means of the image data and an image data stream.

Claims

1. A mobile, maneuverable, particularly calibratable device having a mobile device head, particularly a non-medical mobile device head with a distal end for arrangement relative to a technical body, or a medical, mobile device head having a distal end for arrangement relative to a tissue-like body, particularly having a distal end for insertion or attachment on the body, having:

at least one mobile device head for manual or automatic guidance,

a guide device, wherein the guide device is designed for the purpose of providing navigation information for the guidance of the mobile device head wherein the distal end thereof can be guided in a near environment,

an image data capture device which is designed to capture image data of an environment of the device head, particularly continuously, and to make the same available,

an image data processing device which is designed for the purpose of compiling a map of the environment, and

a navigation device which is designed to indicate at least one position of the device head in the near environment using the map, by means of the image data and an image data stream, in such a manner that the mobile device head can be guided using the map, wherein

a guiding means, with a position reference to the device head, functionally assigned to the same, which is designed to give specifications on the position of the device head in the map with respect to the environment, wherein the environment extends beyond the near environment.

2. A device according to claim 1, characterized in that the position reference of the guiding means to the device head is stationary or moveable in a deterministic way, and particularly can be calibrated.

3. A device according to claim 1, characterized in that the guiding means comprises the image data capture device and/or a further orientation module which is designed to provide a further specification on the position, particularly the pose (position and orientation) and/or movement of the device head with respect to the map.

4. A device according to claim 1, characterized in that the orientation module comprises a movement module and/or acceleration sensor or similar system of sensors, and/or the orientation module comprises at least one lens particularly a target sighting lens and/or guide lens and/or an external lens.

5. A device according to claim 1, characterized in that it is possible to specify a position, particularly a pose and/or movement of the device head in the near environment using the map, in such a manner that a controller and a maneuvering apparatus can guide the mobile device head according to the position, particularly the position and/or orientation and/or movement of the device head, and using the map of the environment.

6. A device according to claim 1, characterized in that the maneuvering apparatus is designed to automatically guide the mobile device head by means of the controller, via a control connection, and the controller is designed for the purpose of navigation of the device head, by means of the guide device, via a data coupling, and particularly the control connection is designed for the purpose of transmitting a TARGET position, particularly a pose, and/or a TARGET movement of the device head, and the data coupling is designed for the transmission of a CURRENT position, particularly a pose, and/or a CURRENT movement of the device head.

7. A device according to claim 1, characterized in that the navigation device has an extended Kalman filter and/or a module for executing a SLAM algorithm.

8. A device according to claim 1, characterized in that the guide device is further designed to guide the at least one mobile device head only using the map, and particularly the guide device has an absolute tracking module, particularly a further sensor system, which can be temporarily activated with limited functionality, or deactivated, and/or can be partially activated or deactivated for the purpose of compiling the map of the near environment.

9. A device according to claim 1, characterized in that the device head is a first mobile device head, and it is possible to guide at least one second mobile device head, particularly a plurality of mobile device heads, using the map, particularly the same, single map.

10. A device according to claim 1, characterized in that the map contains markers, and the guiding means is designed to recognize at least one unknown or known body shape in the environment as a marker, wherein it is possible to determine the relative position, particularly the relative pose, of the device head to a position, particularly a position of the known body shape, and particularly it is possible to determine, simultaneously, a relative position, particularly a relative pose, of a number of body shapes with respect to each other and/or to the device head.

11. A device according to claim 1, characterized in that the map contains markers, wherein objects are inserted into the environment as markers, and these are particularly suited to being detected by the image data capture device, and particularly the guiding means is designed to measure a fixed object a single time, and for multiple uses as a marker, or to measure the same continuously.

12. A device according to claim 1, characterized in that the position, particularly the pose and/or movement, of the device head can be specified using the map relative to a reference point on a body shape or an object in the environment of the device head, wherein the reference point is part of the map of the environment, particularly the near environment.

13. A device according to claim 1, characterized in that a reference point lies outside of the near environment, wherein it is possible to specify a determined relation between the reference point and a map position.

14. A device according to claim 1, characterized in that the guiding means is designed to measure a moving mechanism or a movement kinematics for position deviations, one time, at regular intervals, or continuously.

15. A device according to claim 1, characterized in that the image data processing device is designed for the purpose of identifying a reference point on an object in a visual image, with a fixed point of an auxiliary image following a certain test.

16. A device according to claim 1, characterized in that the image data processing device is designed to register and/or add to a visual image, particularly the map, with an auxiliary image, particularly a computer tomography or magnetic resonance tomography image (CT or MRT image) or a similar image, particularly initially, at regular intervals, or continuously.

17. A device according to claim 1, characterized in that the image data processing device has a module which is designed to recognize target movements, particularly target body movements, particularly target body movements which can be recognized according to a physiological pattern and are particularly rhythmic, and to take the same into account for the compiling of a map of the near environment.

18. A device according to claim 1, characterized in that the environment includes the near environment.

19. A device according to claim 1, characterized in that the environment is disjunct from the near environment.

20. A device according to claim 1, characterized in that the near environment is an operation environment of the distal end of the mobile device head.

21. A device according to claim 1, characterized in that the near environment comprises the image data which is captured in the visual range of a first lens of the image data capture device on the distal end of the mobile device head.

22. A device according to claim 1, characterized in that the environment includes a region which lies in the near environment and beyond the operation environment of the distal end of the mobile device head.

23. A device according to claim 1, characterized in that at least one first and one second region of the environment can be captured by the guiding means and/or the image data capture device, wherein at least the first region is a part of an operation environment in which a movement can be detected, particularly a movement of a body shape and/or a distal end of the device head.

24. A device according to claim 1, characterized in that at least one first and one second region of the environment can be captured by the guiding means and/or the image data capture device, wherein in at least the first captured region it is possible to detect and/or compensate for, by means of the guiding means, errors, failures, or lost signals or similar malfunctions by means of the analysis of at least the second region.

25. A device according to claim 1, characterized in that the guiding means, particularly the image data capture device and/or the orientation module is/are designed to continuously capture the image data of the environment of the device head, and the image data processing device is designed to compile a map of the environment by means of the image data, particularly to specify further details on the position—

particularly the pose and/or movement of the device head with respect to the map in real-time, particularly during an operation.

26. A device according to claim 1, characterized in that the image data capture device has at least one, particularly two, three or another number of lenses, which are designed to simultaneously capture the same region or further regions of the environment particularly regions in a near environment and/or in addition to the near environment by means of image data, and particularly a movement of an instrument which can move, can be determined using at least two lenses.

27. A device according to claim 1, characterized in that a first lens captures first image data and a second lens captures second image data, wherein the first and second image data are captured at the same time, and are offset spatially, or wherein the first and second image data are captured in the same space, and at different times.

28. A device according to claim 1, characterized in that the image data capture device has a sighting lens which sits on a distal end of the device head, wherein the sighting lens is designed to capture image data of a near environment on a distal end of the device head.

29. A device according to claim 1, characterized in that the guiding means, particularly the image data capture device and/or the orientation module, has a guide lens which sits on a guide point—

at a distance from a distal end, particularly on a proximal end of the device head and/or on the guide device wherein the guide lens is designed to capture image data of an environment of the device head, particularly near to the guide point.

34. A device according to claim 2, characterized in that:

the guiding means comprises the image data capture device and/or a further orientation module which is designed to provide a further specification on the position, particularly the pose and/or movement of the device head with respect to the map;

the orientation module comprises a movement module and/or acceleration sensor or similar system of sensors, and/or the orientation module comprises at least one lens, particularly a target sighting lens and/or guide lens and/or an external lens;

it is possible to specify a position, particularly a pose and/or movement of the device head in the near environment using the map, in such a manner that a controller and a maneuvering apparatus can guide the mobile device head according to the position, particularly the position and/or orientation and/or movement of the device head, and using the map of the environment;

the maneuvering apparatus is designed to automatically guide the mobile device head by means of the controller, via a control connection, and the controller is designed for the purpose of navigation of the device head, by means of the guide device, via a data coupling, and particularly the control connection is designed for the purpose of transmitting a TARGET position, particularly a pose, and/or a TARGET movement of the device head, and the data coupling is designed for the transmission of a CURRENT position, particularly a pose, and/or a CURRENT movement of the device head;

the navigation device has an extended Kalman filter (EKF) and/or a module for executing a SLAM algorithm;

the guide device is further designed to guide the at least one mobile device head only using the map, and particularly the guide device has an absolute tracking module, particularly a further sensor system, which can be temporarily activated with limited functionality, or deactivated, and/or can be partially activated or deactivated for the purpose of compiling the map of the near environment;

the device head is a first mobile device head, and it is possible to guide at least one second mobile device head, particularly a plurality of mobile device heads, using the map, particularly the same, single map;

the map contains markers, and the guiding means is designed to recognize at least one unknown or known body shape in the environment as a marker, wherein it is possible to determine the relative position, particularly the relative pose, of the device head to a position, particularly a position of the known body shape, and particularly it is possible to determine, simultaneously, a relative position, particularly a relative pose, of a number of body shapes with respect to each other and/or to the device head;

the map contains markers, wherein objects are inserted into the environment as markers, and these are particularly suited to being detected by the image data capture device, and particularly the guiding means is designed to measure a fixed object a single time, and for multiple uses as a marker, or to measure the same continuously;

the position, particularly the pose and/or movement, of the device head can be specified using the map relative to a reference point on a body shape or an object in the environment of the device head, wherein the reference point is part of the map of the environment, particularly the near environment;

a reference point lies outside of the near environment, wherein it is possible to specify a determined relation between the reference point and a map position;

the guiding means is designed to measure a moving mechanism or a movement kinematics for position deviations, one time, at regular intervals, or continuously;

the image data processing device is designed for the purpose of identifying a reference point on an object in a visual image, with a fixed point of an auxiliary image following a certain test;

the image data processing device is designed to register and/or add to a visual image, particularly the map, with an auxiliary image, particularly a computer tomography or magnetic resonance tomography image (CT or MRT image) or a similar image, particularly initially, at regular intervals, or continuously;

the image data processing device has a module which is designed to recognize target movements, particularly target body movements, particularly target body movements which can be recognized according to a physiological pattern and are particularly rhythmic, and to take the same into account for the compiling of a map of the near environment;

the environment includes the near environment;

the environment is disjunct from the near environment;

the near environment is an operation environment of the distal end of the mobile device head;

the near environment comprises the image data which is captured in the visual range of a first lens of the image data capture device on the distal end of the mobile device head;

the environment includes a region which lies in the near environment and beyond the operation environment of the distal end of the mobile device head;

at least one first and one second region of the environment can be captured by the guiding means and/or the image data capture device, wherein at least the first region is a part of an operation environment in which a movement can be detected, particularly a movement of a body shape and/or a distal end of the device head;

at least one first and one second region of the environment can be captured by the guiding means and/or the image data capture device, wherein in at least the first captured region it is possible to detect and/or compensate for, by means of the guiding means, errors, failures, or lost signals or similar malfunctions by means of the analysis of at least the second region;

the guiding means, particularly the image data capture device and/or the orientation module is/are designed to continuously capture the image data of the environment of the device head, and the image data processing device is designed to compile a map of the environment by means of the image data, particularly to specify further details on the position—

particularly the pose and/or movement of the device head with respect to the map—

in real-time, particularly during an operation;

the image data capture device has at least one, particularly two, three or another number of lenses, which are designed to simultaneously capture the same region or further regions of the environment—

particularly regions in a near environment and/or in addition to the near environment—

by means of image data, and particularly a movement of an instrument which can move, can be determined using at least two lenses;

a first lens captures first image data and a second lens captures second image data, wherein the first and second image data are captured at the same time, and are offset spatially, or wherein the first and second image data are captured in the same space, and at different times;

the image data capture device has a sighting lens which sits on a distal end of the device head, wherein the sighting lens is designed to capture image data of a near environment on a distal end of the device head; and

the guiding means, particularly the image data capture device and/or the orientation module, has a guide lens which sits on a guide point—

at a distance from a distal end, particularly on a proximal end of the device head and/or on the guide device—

wherein the guide lens is designed to capture image data of an environment of the device head, particularly near to the guide point.

30. A method for the maneuvering, and particularly calibration, of a device having a mobile device head, particularly a non-medical mobile device head having a distal end for arrangement relative to a technical body, or a medical, mobile device head having a distal end for arrangement relative to a tissue-like body, particularly having a distal end for insertion or attachment on the body, having the steps:

manual or automatic guidance of the mobile device head,

provision of navigation information for the guidance of the mobile device head, wherein the distal end thereof is guided in a near environment,

capturing and provision of image data of an environment of the device head, particularly continuously,

compiling of a map of the environment by means of the image data,

specifying at least one position of the device head in the near environment using the map, by means of the image data and an image data stream, in such a manner that the mobile device head can be guided using the map, wherein

it is possible to provide details of the position of the device head in the map with respect to the environment, with the position reference to the device head, wherein the environment extends beyond the near environment.

31. A method according to claim 30, characterized in that the position reference to the device head is stationary or moves in a determined manner, and particularly is calibrated.

32. A method according to claim 30, characterized in that a further specification on the position, particularly the pose and/or the movement of the device head is provided in reference to the map.

33. A method according to claim 30, characterized in that the mobile device head is guided automatically, wherein a TARGET position, particularly pose, and/or a TARGET movement of the device head is transmitted, and a CURRENT position, particularly pose, and/or CURRENT movement of the device head is transmitted.

35. A method according to claim 31, characterized in that:

a further specification on the position, particularly the pose (position and/or orientation) and/or the movement of the device head is provided in reference to the map; and

the mobile device head is guided automatically, wherein a TARGET position, particularly pose, and/or a TARGET movement of the device head is transmitted, and a CURRENT position, particularly pose, and/or CURRENT movement of the device head is transmitted.