SURGICAL ROBOT SYSTEM AND METHOD OF CONTROLLING THE SAME

• US 20140288413A1
• Filed: 08/13/2013
• Published: 09/25/2014
• Est. Priority Date: 03/21/2013
• Status: Active Grant

• Current Assignee: Samsung Electronics Co. Ltd.
• Original Assignee: Samsung Electronics Co. Ltd.
• Inventors: Hwang, Won Jun, Roh, Kyung Shik, Yoon, Suk June, Hyung, Seung Yong
• Granted Patent: US 9,615,890 B2
• Time in Patent Office: Days
• US Class Current: 600/424
• CPC Class Codes: A61B 2090/364 Correlation of different images or relation of image positions in respect to the body
• CPC Class Codes: A61B 2090/367 creating a 3D dataset from 2D images using position information
• CPC Class Codes: A61B 34/30 Surgical robots
• CPC Class Codes: A61B 34/37 Master-slave robots A61B34/35 takes precedence
• CPC Class Codes: A61B 90/361 Image-producing devices, e.g. surgical cameras

Abstract

A surgical robot system includes a slave system to perform a surgical operation on a patient and an imaging system that includes an image capture unit including a plurality of cameras to acquire a plurality of affected area images, an image generator detecting an occluded region in each of the affected area images acquired by the plurality of cameras, removing the occluded region therefrom, warping each of the affected area images from which the occluded region is removed, and matching the affected area images to generate a final image, and a controller driving each of the plurality of cameras of the image capture unit to acquire the plurality of affected area images and inputting the acquired plurality of affected area images to the image generator to generate a final image.

Claims

1. A surgical robot system comprising:

a slave system to perform a surgical operation on a patient; and

an imaging system comprising;

an image capture unit comprising a plurality of cameras to acquire a plurality of affected area images;

an image generator detecting an occluded region in each of the affected area images acquired by the plurality of cameras, removing the occluded region therefrom, warping each of the affected area images from which the occluded region is removed, and matching the affected area images to generate a final image; and

a controller driving each of the plurality of cameras of the image capture unit to acquire the plurality of affected area images and inputting the acquired plurality of affected area images to the image generator to generate a final image.

2. The surgical robot system according to claim 1, wherein the image generator comprises:

an image processor detecting the occluded region by image processing of each of the affected area images acquired by the plurality of cameras and removing the detected occluded region;

an image warping unit warping each of the affected area images from which the occluded region is removed; and

an image matching unit matching the warped affected area images to generate the final image in which the occluded region is restored.

3. The surgical robot system according to claim 1, wherein the image capture unit further comprises a support member provided with the plurality of cameras.

4. The surgical robot system according to claim 3, wherein the plurality of cameras are mounted on the support member to be spaced apart from each other by a predetermined distance.

5. The surgical robot system according to claim 3, wherein the image capture unit further comprises moving units respectively moving the plurality of cameras along the support member.

6. The surgical robot system according to claim 5, wherein the controller estimates depth information from the plurality of cameras to the detected occluded region, respectively moves the plurality of cameras to corresponding positions according to the estimated depth information using the moving units, and re-acquires the plurality of affected area images.

7. The surgical robot system according to claim 1, wherein the camera comprises a depth camera.

8. The surgical robot system according to claim 1, further comprising a master system to control the surgical operation of the slave system.

9. The surgical robot system according to claim 8, wherein the imaging system further comprises a communication unit, and the controller transmits the final image generated by the image generator to the slave system or the master system via the communication unit.

10. A method of controlling a surgical robot, the method comprising:

acquiring a plurality of affected area images using a plurality of cameras;

detecting an occluded regions present in each of the acquired affected area images;

removing the detected occluded regions from each of the affected area images;

warping each of the affected area images from which the occluded regions are removed; and

generating a final image by matching the warped affected area images.

11. The method according to claim 10, wherein the detected occluded regions are disposed at different positions in the affected area images.

12. The method according to claim 10, wherein the occluded region is a region at which a surgical instrument is located.

13. The method according to claim 12, wherein the detecting of the occluded region is performed by attaching a marker to the surgical instrument, and detecting the marker from each of the affected area images.

14. The method according to claim 12, wherein the detecting of the occluded region is performed by detecting state data of each joint of a robot arm provided with the surgical instrument.

15. The method according to claim 12, wherein the detecting of the occluded region is performed by extracting a blob closest to the camera from the each of affected area images and spreading the extracted blob using a region growing method.

16. The method according to claim 10, wherein the detecting of the occluded region is performed by estimating depth information from the camera to the occluded region.

17. The method according to claim 16, wherein the estimating of the depth information to the occluded region is performed by use of sum of absolute differences (SAD) or sum of squared differences (SSD).

18. The method according to claim 16, wherein the estimating of the depth information to the occluded region is performed by use of a depth sensor attached to the camera.

19. The method according to claim 16, further comprising determining whether the estimated depth information is within a predetermined range after the estimating of the depth information to the occluded region.

20. The method according to claim 19, further comprising:

removing the detected occluded region from each of the affected area images when the depth information is within the predetermined range;

or

moving the plurality of cameras to positions corresponding to the estimated depth information and re-acquiring the plurality of affected area images when the depth information is not within the predetermined range,

after the determining whether the estimated depth information is within a predetermined range.