System and method for smoke detection during anatomical surgery

• US 9,805,472 B2
• Filed: 09/23/2015
• Issued: 10/31/2017
• Est. Priority Date: 02/18/2015
• Status: Active Grant

• Current Assignee: Sony Corporation (Sony Group Corp.)
• Original Assignee: Sony Corporation (Sony Group Corp.)
• Inventors: Chou, Chen-Rui, Liu, Ming-Chang
• Primary Examiner(s): Liew, Alex
• Application Number:
  US14/862,437
• Publication Number:
  US 20160239967A1
• Time in Patent Office: 769 Days
• Field of Search: 382128-134
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/10068 Endoscopic image
• CPC Class Codes: G06T 2207/20076 Probabilistic image processing
• CPC Class Codes: G06T 2207/20224 Image subtraction
• CPC Class Codes: G06T 2207/30004 Biomedical image processing
• CPC Class Codes: G06T 2207/30168 Image quality inspection
• CPC Class Codes: G06T 7/11 Region-based segmentation
• CPC Class Codes: G06T 7/254 involving subtraction of images

Abstract

Various aspects of a system and method for smoke detection during an anatomical surgery are disclosed herein. In accordance with an embodiment of the disclosure, the method is implementable in a surgical scene analysis engine, which is communicatively coupled to an image-capturing device that captures one or more video frames. The method includes the estimation of a partially visible region in a current video frame, based on a temporal difference between the current video frame and a previous video frame from the one or more video frames. Thereafter, one or more candidate pixels are detected in the estimated partially visible region in the current video frame. Further, a smoke region is determined in the partially visible region, based on pruning of one or more candidate pixels.

Claims

1. An apparatus, comprising:

circuitry configured to;

capture at least one video frame of an anatomical region of a user;

estimate a partially visible region in a current video frame of said anatomical region based on a temporal difference between said current video frame and a previous video frame of said anatomical region;

detect at least one candidate pixel in said estimated partially visible region in said current video frame based on a difference of average intensity of corresponding image blocks of said current video frame and said previous video frame;

determine at least one false-positive candidate pixel from said detected at least one candidate pixel;

prune said at least one false-positive candidate pixel from said detected at least one candidate pixel; and

determine a smoke region in said partially visible region based on said pruned at least one candidate pixel.

2. The apparatus of claim 1, wherein said at least one candidate pixel corresponds to at least one of a smoke block, a tissue block, or a non-tissue block.

8. The apparatus of claim 1, wherein said estimation of said partially visible region is based on an entropy of a histogram of said temporal difference between said current video frame and said previous video frame.

9. The apparatus of claim 1, wherein said circuitry is further configured to adjust at least one image-capture setting to capture said at least one video frame, based on said determination of said smoke region, wherein said at least one image-capture setting comprises at least one of an automatic exposure, an automatic focus, an automatic white balance, or an automatic illumination.

10. The apparatus of claim 1, further comprising:

a display screen,

wherein said circuitry is further configured to display said current video frame via a user interface rendered on the display screen, and wherein said determined smoke region is at least masked or highlighted in said displayed current video frame.

11. The apparatus of claim 1, wherein said circuitry is further configured to generate a notification indicative of said determination of said smoke region, and wherein said notification corresponds to at least one of an audio alert, a textual alert, a visual alert, or a haptic alert.

3. An apparatus, comprising:

circuitry configured to;

capture at least one video frame;

estimate a partially visible region in a current video frame based on a temporal difference between said current video frame and a previous video frame;

detect at least one candidate pixel in said estimated partially visible region in said current video frame based on a difference of average intensity of corresponding image blocks of said current video frame and said previous video frame;

prune at least one false-positive candidate pixel from said detected at least one candidate pixel; and

determine a smoke region in said partially visible region based on said pruned at least one false-positive candidate pixel from said detected at least one candidate pixel,

wherein said at least one false-positive candidate pixel comprise at least one of a first set of pixels in a motion-blurred region in said current video frame or a second set of pixels in a halo-effected region in said current video frame.

4. The apparatus of claim 3, wherein said motion-blurred region corresponds to a region in said current video frame that encompasses a movement of at least one of the apparatus, a non-tissue object, or a tissue portion.

5. The apparatus of claim 3, wherein said circuitry is further configured to detect said motion-blurred region by use of at least one of an optical flow based technique or an artefact detection technique.

6. The apparatus of claim 5, wherein said circuitry is further configured to identify at least one first edge pixel in said current video frame and at least one second edge pixel in said previous video frame by use of an edge detection technique, wherein said motion-blurred region is detected based at least on a difference in location of said at least one first edge pixel from said at least one second edge pixel.

7. The apparatus of claim 3, wherein said circuitry is further configured to detect said halo-effected region based on a region growth rate estimation technique.

12. A method, comprising:

capturing at least one video frame of an anatomical region of a user;

estimating a partially visible region in a current video frame of said anatomical region based on a temporal difference between said current video frame and a previous video frame of said anatomical region;

detecting at least one candidate pixel in said estimated partially visible region in said current video frame based on a difference of average intensity of corresponding image blocks of said current video frame and said previous video frame;

determining at least one false-positive candidate pixel from said detected at least one candidate pixel;

pruning said at least one false-positive candidate pixel from said detected at least one candidate pixel; and

determining a smoke region in said partially visible region based on pruning of said detected at least one candidate pixel.

13. The method of claim 12, wherein said at least one candidate pixel corresponds to at least one of a smoke block, a tissue block, or a non-tissue block.

14. The method of claim 12, further comprising determining at least one false-positive candidate pixel from said detected at least one candidate pixel, wherein said pruning of said detected at least one candidate pixel further comprises removal of said at least one false-positive candidate pixel from said detected at least one candidate pixel.

15. The method of claim 14, wherein said at least one false-positive candidate pixel comprise at least one of a first set of pixels in a motion-blurred region in said current video frame or a second set of pixels in a halo-effected region in said current video frame.

16. The method of claim 12, further comprising adjusting at least one image-capture setting to capture said at least one video frame, based on said determination of said smoke region,

wherein said at least one image-capture setting comprises at least one of an automatic exposure, an automatic focus, an automatic white balance, or an automatic illumination.

17. The method of claim 12, further comprising displaying said current video frame via a user interface rendered on a display screen, wherein said determined smoke region is at least masked or highlighted in said displayed current video frame.

18. The method of claim 12, further comprising generating a notification indicative of said determination of said smoke region, wherein said notification corresponds to at least one of an audio alert, a textual alert, a visual alert, or a haptic alert.

19. A non-transitory computer-readable medium having stored thereon, computer-executable instructions for causing a computer to execute operations, the operations comprising:

capturing at least one video frame of an anatomical region of a user;

estimating a partially visible region in a current video frame of said anatomical region based on a temporal difference between said current video frame and a previous video frame of said anatomical region;

detecting at least one candidate pixel in said estimated partially visible region in said current video frame based on a difference of average intensity of corresponding image blocks of said current video frame and said previous video frame;

determining at least one false-positive candidate pixel from said detected at least one candidate pixel;

pruning said at least one false-positive candidate pixel from said detected at least one candidate pixel; and

determining a smoke region in said partially visible region based on pruning of said detected at least one candidate pixel.