System and method for multi-resolution sharpness transport across color channels

• US 9,697,588 B2
• Filed: 11/15/2010
• Issued: 07/04/2017
• Est. Priority Date: 11/15/2010
• Status: Active Grant

• Current Assignee: Intuitive Surgical Operations, Inc. (Intuitive Surgical Holdings, LLC)
• Original Assignee: Intuitive Surgical Operations, Inc. (Intuitive Surgical Holdings, LLC)
• Inventors: Scott, David D., Zhao, Wenyi, DiMaio, Simon P.
• Primary Examiner(s): Kelley, Christopher S
• Assistant Examiner(s): Tarko, Asmamaw G
• Application Number:
  US12/946,634
• Publication Number:
  US 20120120245A1
• Time in Patent Office: 2,423 Days
• Field of Search: 348222, 382240
• CPC Class Codes: G06T 2207/10024 Color image
• CPC Class Codes: G06T 2207/20016 Hierarchical, coarse-to-fine, multiscale or multiresolution image processing; Pyramid transform
• CPC Class Codes: G06T 2207/20212 Image combination
• CPC Class Codes: G06T 5/10 using non-spatial domain filtering
• CPC Class Codes: G06T 5/73 Deblurring; Sharpening

Abstract

Provided are a system and method for image sharpening is provided that involves capturing an image, and then decomposing the image into a plurality of image-representation components, such as RGB components for example. Each image-representation component is transformed to obtain an unsharpened multi-resolution representation for each image-representation component. A multi-resolution representation includes a plurality of transformation level representations. Sharpness information is transported from an unsharpened transformation level representation of a first one of the image-representation components to a transformation level representation of an unsharpened multi-resolution representation of a second one of the image-representation components to create a sharpened multi-resolution representation of the second one of the image-representation components. The sharpened multi-resolution representation of the second one of the image-representation components is then transformed to obtain a sharpened image. The improved and sharpened image may then be displayed.

Claims

1. An image sharpening method comprising:

capturing an image;

decomposing, by an image system, the image into a plurality of image-representation components;

transforming, by the image system, each image-representation component of the plurality of image-representation components to obtain an unsharpened multi-resolution representation for each image-representation component, wherein a multi-resolution representation comprises a plurality of transformation level representations;

transporting, by the image system, sharpness information from an unsharpened transformation level representation of a first one of the plurality of image-representation components to a same transformation level representation of an unsharpened multi-resolution representation of a second one of the plurality of image-representation components to create a sharpened multi-resolution representation of the second one of the image-representation components, wherein the transported sharpness information modifies sharpness information in the same transformation level representation of the unsharpened multi-resolution representation of the second one of the plurality of image-representation components;

transforming, by the image system, the sharpened multi-resolution representation of the second one of the plurality of image-representation components to obtain a sharpened image; and

displaying the sharpened image on a display device.

2. The method of claim 1:

wherein the capturing the image comprises illuminating a scene to be imaged with narrow band illumination.

3. The method of claim 1:

wherein the capturing the image comprises illuminating a scene to be imaged with infrared illumination.

4. The method of claim 1:

wherein the capturing the image comprises illuminating a scene to be imaged with wide band illumination.

5. The method of claim 1:

wherein the capturing the image comprises illuminating a scene to be imaged with X-Ray illumination.

6. The method of claim 1:

wherein the capturing the image comprises capturing the image with a single image capture area on an integrated circuit device.

7. The method of claim 1:

wherein the capturing the image comprises capturing the image with a plurality of integrated circuit devices in a single camera.

8. The method of claim 1:

wherein the capturing the image comprises capturing each image component of the image with a different camera in a plurality of cameras.

9. The method of claim 8, the capturing the image further comprising:

aligning the image components captured with the plurality of cameras.

10. The method of claim 1:

wherein the image comprises near infrared spectrum features and visible spectrum features.

11. The method of claim 1:

wherein the image comprises X-Ray.

12. The method of claim 1:

wherein decomposing the image into a plurality of image-representation components comprises performing a color space transform.

13. The method of claim 12:

wherein performing the color space transform comprises transforming RGB image-representation components to YCbCr image-representation components.

14. The method of claim 1:

wherein the plurality of image-representation components comprises RGB image-representation components.

15. The method of claim 1:

wherein one of the plurality of image-representation components comprises information from a near infrared spectrum feature of the image and another one of the plurality of image-representation components comprises information from a visible spectrum feature of the image.

16. The method of claim 1:

wherein the transformation level representations comprise Laplacian representations.

17. The method of claim 1:

wherein the transformation level representations comprise wavelet representations.

18. The method of claim 1 further comprising:

transporting sharpness information from a second unsharpened transformation level representation of the first one of the plurality of image-representation components to a second transformation level representation of the unsharpened multi-resolution representation of the second one of the plurality of image-representation components to create the sharpened multi-resolution representation of the second one of the plurality of image-representation components.

19. The method of claim 1:

wherein the sharpness information is image-driven.

20. The method of claim 19:

wherein the image-driven sharpness information is gradient-based.

21. The method of claim 19:

wherein the image-driven sharpness information is based on local statistics.

22. The method of claim 21:

wherein the local statistics comprise kurtosis and/or eigenvalues.

23. The method of claim 1:

wherein the sharpness information comprises sharpest sharpness information selected from each of the unsharpened multi-resolution representations for each image-representation component of the plurality of image-representation components.

24. The method of claim 1:

wherein the sharpened image comprises an RGB image.

25. The method of claim 1, further comprising:

transforming the unsharpened multi-resolution representation for the first one of the plurality of image-representation components and the sharpened multi-resolution representation of the second one of the plurality of image-representation components to obtain the sharpened image.

26. The method of claim 25:

wherein the sharpened image comprises an RGB image.

27. The method of claim 1:

wherein the sharpness information in the transformation level of the sharpened multi-resolution representation of the second one of the plurality of image-representation components comprises a weighted sum that includes the sharpness information from the unsharpened multi-resolution representation for the first one of the image representation components and sharpness information from the unsharpened multi-resolution representation for the second one of the image representation components.

28. The method of claim 1, further comprising:

selecting, by the image system, a first sharpness state or a second sharpness state of the sharpened image in response to a sharpness selection input;

wherein for the first sharpness state, sharpness information is transported from a first number of unsharpened transformation level representations; and

wherein for the second sharpness state, sharpness information is transported from a second number, different from the first number, of unsharpened transformation level representations.

29. The method of claim 1:

wherein the transformation level representations comprise Laplacian transformation level representations; and

wherein the sharpness information comprises an absolute value of pixel values in the unsharpened transformation level representation of the first one of the plurality of image-representation components.