SYSTEM AND METHOD FOR GENERATING A MULTI-DIMENSIONAL IMAGE

• US 20090290811A1
• Filed: 05/05/2009
• Published: 11/26/2009
• Est. Priority Date: 05/23/2008
• Status: Active Grant

• Current Assignee: Samsung Electronics Co. Ltd.
• Original Assignee: Samsung Electronics Co. Ltd.
• Inventors: Imai, Francisco H.
• Granted Patent: US 8,442,355 B2
• Time in Patent Office: Days
• US Class Current: 382/285
• CPC Class Codes: G06T 2200/04 involving 3D image data
• CPC Class Codes: G06T 2207/10028 Range image; Depth image; 3D point clouds
• CPC Class Codes: G06T 2207/20221 Image fusion; Image merging
• CPC Class Codes: G06T 3/4061 by injecting details from different spectral ranges
• CPC Class Codes: G06T 7/11 Region-based segmentation

Abstract

A system and method for generating a multi-dimensional image of an object in a scene is disclosed. One inventive aspect includes a spectral estimation module configured to convert a two-dimensional (2D) high-resolution light intensity image of the scene to a spectral-augmented image of a selected channel. The system further includes a high-resolution depth image generation module configured to generate a high-resolution depth image of the object based on a three-dimensional (3D) low-resolution depth image of the scene and the spectral-augmented image.

Claims

1. A method of generating a multi-dimensional image of an object in a scene, the method comprising:

providing a three-dimensional (3D ) low-resolution depth image of the scene and a two-dimensional (2D) high-resolution light intensity image of the scene;

converting the 2D high-resolution light intensity image to a spectral-augmented image of a selected channel;

segmenting the object from the spectral-augmented image; and

generating a high-resolution depth image of the object based at least in part on the 3D low-resolution depth image and the spectral-augmented image.

2. The method of claim 1, further comprising generating a 3D high-resolution spectral image of the object based on the high-resolution depth image and the spectral-augmented image, the 3D spectral image comprising volume and spectral information.

3. The method of claim 2, further comprising augmenting a user'"'"'s direct vision of the scene by superimposing the 3D high-resolution spectral image on the direct vision.

4. The method of claim 1, wherein the object is a human body part under a medical operation or examination by a physician, and wherein the high-resolution depth image is used to provide an augmented vision for the physician.

5. The method of claim 1, wherein the object comprises non-rigid human or animal organ tissues.

6. The method of claim 1, wherein the 2D light intensity image comprises two, three, or four channels.

7. The method of claim 1, wherein the 2D light intensity image comprises electronic bandpass wavelength selection.

8. The method of claim 1, wherein the channel of the spectral-augmented image is variably selected to make differences between the object and its surrounding region more noticeable in the spectral-augmented image than in the 2D light intensity image.

9. The method of claim 1, wherein the channel is selected based on a user input depending at least on spectral reflectance properties of the object.

10. The method of claim 1, wherein converting the 2D high-resolution light intensity image to a spectral-augmented image of a selected channel comprises:

reconstructing a spectral image having at least three channels from the 2D high-resolution light intensity image; and

filtering the spectral image to generate an image of the selected channel.

11. The method of claim 1, wherein generating a high-resolution depth image of the object comprises:

generating a high-resolution depth image of the scene by interpolating depth values from the low-resolution depth image; and

segmenting the object from the high-resolution depth image.

12. The method of claim 1, wherein each of the 3D low-resolution depth image and the 2D high-resolution light intensity image comprises a plurality of images in a video, and the method is performed in real time to track the object in the scene across the images.

13. A system for generating a multi-dimensional image of an object in a scene, the system comprising:

means for providing a three-dimensional (3D ) low-resolution depth image of the scene and a two-dimensional (2D) high-resolution light intensity image of the scene;

means for converting the 2D high-resolution light intensity image to a spectral-augmented image of a selected channel; and

means for generating a high-resolution depth image of the object based at least in part on the 3D low-resolution depth image and the spectral-augmented image.

14. The system of claim 13, further comprising means for segmenting the object from the spectral-augmented image.

15. A system for generating a multi-dimensional image of an object in a scene, the system comprising:

a spectral estimation module configured to convert a two-dimensional (2D) high-resolution light intensity image of the scene to a spectral-augmented image of a selected channel; and

a high-resolution depth image generation module configured to generate a high-resolution depth image of the object based on a three-dimensional (3D ) low-resolution depth image of the scene and the spectral-augmented image.

16. The system of claim 15, further comprising an object segmentation module configured to segment the object from the spectral-augmented image.

17. The system of claim 15, further comprising a depth imaging module configured to generate the low-resolution 3D depth image of the scene.

18. The system of claim 15, further comprising an array imaging module configured to generate the 2D high-resolution light intensity image of the scene, the array imaging module having no more than four channels.

19. The system of claim 18, wherein the array imaging module comprises a RGB camera.

20. The system of claim 18, wherein the array imaging module comprises a black/white camera configured to take images while varying illumination.

21. The system of claim 15, further comprising an augmented volume generation module configured to generate a 3D high-resolution spectral image of the object based on the high-resolution depth image and the spectral-augmented image, the 3D spectral image comprising volume and spectral information.

22. The system of claim 21, further comprising a vision augmentation module configured to augment a user'"'"'s direct vision of the scene by superimposing the 3D high-resolution spectral image on the direct vision.

23. The system of claim 15, wherein the spectral estimation module is configured to select a channel based on a user input indicating the object to be segmented.