THREE-DIMENSIONAL SCAN RECOVERY

• US 20070171220A1
• Filed: 05/16/2006
• Published: 07/26/2007
• Est. Priority Date: 01/20/2006
• Status: Active Grant

• Current Assignee: Medit Corp.
• Original Assignee: 3M Innovative Properties Company (3M Company)
• Inventors: Kriveshko, Ilya
• Granted Patent: US 7,940,260 B2
• Time in Patent Office: Days
• US Class Current: 345/419
• CPC Class Codes: A61C 9/0046 Data acquisition means or methods
• CPC Class Codes: A61C 9/0086 Acoustic means or methods
• CPC Class Codes: G01B 11/24 for measuring contours or curvatures
• CPC Class Codes: G06T 17/00 Three dimensional [3D] modelling, e.g. data description of 3D objects
• CPC Class Codes: G06T 2200/04 involving 3D image data
• CPC Class Codes: G06T 2200/24 involving graphical user interfaces [GUIs]
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/10028 Range image; Depth image; 3D point clouds
• CPC Class Codes: G06T 2207/30036 Dental; Teeth
• CPC Class Codes: G06T 7/30 Determination of transform parameters for the alignment of images, i.e. image registration

Abstract

A scanning system that acquires three-dimensional images as an incremental series of fitted three-dimensional data sets is improved by testing for successful incremental fits in real time and providing a variety of visual user cues and process modifications depending upon the relationship of newly acquired data to previously acquired data. The system may be used to aid in error-free completion of three-dimensional scans. The methods and systems described herein may also usefully be employed to scan complex surfaces including occluded or obstructed surfaces by maintaining a continuous three-dimensional scan across separated subsections of the surface. In one useful dentistry application, a full three-dimensional surface scan may be obtained for two dental arches in occlusion.

Claims

1. A method comprising:

acquiring three-dimensional surface data from a subject as a sequence of three-dimensional images, each one of the sequence of three-dimensional images fitted to at least one previous one of the sequence of three-dimensional images;

acquiring an image set for a next three-dimensional image;

when the image set can be converted into the next three-dimensional image and the next three-dimensional image can be fitted to the three-dimensional surface data, superimposing the three-dimensional surface data and the next three-dimensional image onto a two-dimensional image of the subject in a display; and

when the image set cannot be converted into the next three-dimensional image or the next three-dimensional image cannot be fitted to the three-dimensional surface data, entering a recover mode comprising the steps of;

superimposing a current two-dimensional image of the subject onto a previous two-dimensional image of the subject in the display, the previous two-dimensional image representing a view of the subject from a position from which one of the sequence of three-dimensional images was acquired, and the current two-dimensional image representing a view of the subject from a position from which the image set was acquired;

acquiring at least one subsequent three-dimensional image; and

test fitting the at least one subsequent three-dimensional image to one or more of the sequence of three-dimensional images.

2. The method of claim 1 further comprising adding the next three-dimensional image to the three-dimensional surface data when the image set can be converted into the next three-dimensional image and the next three-dimensional image can be fit to the three-dimensional surface data.

3. The method of claim 1 wherein the previous two-dimensional image and the current two-dimension image include video frames.

4. The method of claim 1 further comprising manually entering the recover mode in response to a user input.

5. The method of claim 1 further comprising automatically entering the recover mode in response to a detection of a lost acquisition.

6. The method of claim 5 wherein the detection of the lost acquisition includes a detection of a motion blur.

7. The method of claim 5 wherein the detection of the lost acquisition includes a detection of an insufficient recovery of three-dimensional data.

8. The method of claim 1 wherein the recover mode includes the step of highlighting a region on the subject for attempting a recovery.

9. The method of claim 8 wherein the region includes a plurality of previous ones of the sequence of three-dimensional images.

10. The method of claim 1 wherein the recover mode includes the step of test fitting the at least one subsequent three-dimensional image to one or more additional images acquired during the recover mode and test fitting includes selecting a reference image from the sequence of three-dimensional images based on a suitability for test fitting.

11. The method of claim 10 further comprising determining the suitability for test fitting based upon an entropy of image data in the previous three-dimensional image.

12. The method of claim 10 further comprising determining the suitability for test fitting based upon an amount of three-dimensional data in the previous three-dimensional image.

13. The method of claim 10 further comprising determining the suitability for test fitting based upon a placement of three-dimensional data within a range and a field of view of an imaging device.

14. The method of claim 10 further comprising determining the suitability for test fitting based upon a temporal proximity of the at least one subsequent three-dimensional image to one or more of the sequence of three-dimensional images.

15. The method of claim 10 further comprising determining the suitability for test fitting based upon a spatial proximity of the at least one subsequent three-dimensional image to one or more of the sequence of three-dimensional images.

16. The method of claim 1 wherein the recover mode includes the step of manually selecting one or more of the sequence of three-dimensional images.

17. The method of claim 1 wherein the recover mode includes selecting a reference image from a beginning or an end of the sequence of three-dimensional images.

18. A method comprising:

providing three-dimensional surface data for a subject;

acquiring an image set for a three-dimensional image of the subject;

when the image set cannot be fitted to the three-dimensional surface data, entering a landing mode comprising the steps of;

superimposing a current two-dimensional image of the subject onto a previous two-dimensional image of the subject in the display, the current two-dimensional image representing a view of the subject from a position from which the image set was acquired, and the previous two-dimensional image representing a view of the subject from which at least a portion of the three-dimensional surface data was acquired;

acquiring at least one subsequent three-dimensional image;

fitting the at least one subsequent three-dimensional image to a second three-dimensional surface reconstruction;

test fitting the at least one subsequent three-dimensional image to the three-dimensional surface data; and

when the image set can be fitted to the three-dimensional surface data, superimposing the three-dimensional surface data and the next three-dimensional image onto a two-dimensional image of the subject in a display and adding the second three-dimensional surface reconstruction to the three-dimensional surface data.

19. The method of claim 18 wherein the previous two-dimensional image and the current two-dimension image include video frames.

20. The method of claim 18 further comprising manually entering the landing mode in response to a user input.

21. The method of claim 18 further comprising, in the landing mode, highlighting a region on the subject for attempting a landing.

22. The method of claim 21 wherein the region includes a portion of the three-dimensional surface data.

23. The method of claim 18 wherein test fitting includes selecting a reference image from the three-dimensional surface data based on a suitability for test fitting.

24. The method of claim 23 wherein the suitability is based on an entropy of image data in the previous three-dimensional image.

25. The method of claim 23 wherein the suitability is based on an amount of three-dimensional data in the previous three-dimensional image.

26. The method of claim 23 wherein the suitability is based on a placement of three-dimensional data within a range and a field of view of an imaging device.

27. The method of claim 23 wherein the suitability is based on a spatial proximity of the image set to a portion of the three-dimensional surface data.

28. The method of claim 18 further comprising manually selecting a region of the three-dimensional surface data for test fitting.

29. The method of claim 18 wherein test fitting includes selecting a reference image from a beginning or an end of a sequence of three-dimensional images used to construct the three-dimensional surface data.

30. The method of claim 18 wherein the previous two-dimensional image and the current two-dimension image include video frames.

31. The method of claim 18 further comprising manually entering the recover mode in response to a user input.

32. The method of claim 18 further comprising manually selecting a region from the three-dimensional surface data for test fitting.

33. A system comprising;

a scanning device that acquires three-dimensional surface data from a subject as a sequence of three-dimensional images, each one of the sequence of three-dimensional images fitted to at least one previous one of the sequence of three-dimensional images, the device including an acquisition mode in which one or more additional three-dimensional images are acquired and added to the three-dimensional surface data, and the device including a recovery mode in which one or more additional three-dimensional images are test fit to one or more of the sequence of three-dimensional images to recover the acquisition mode; and

a display adapted to provide one or more visual indicators distinguishing between the acquisition mode and the recovery mode.

34. The system of claim 33, the display further adapted to provide guidance to an operator in manipulating the subject in the recovery mode.

35-101. -101. (canceled)