Navigating among images of an object in 3D space

• US 9,937,022 B2
• Filed: 01/04/2009
• Issued: 04/10/2018
• Est. Priority Date: 01/04/2008
• Status: Active Grant

• Current Assignee: Medit Corp.
• Original Assignee: 3M Innovative Properties Company (3M Company)
• Inventors: Kriveshko, Ilya A., Dedkov, Dmitriy A.
• Primary Examiner(s): Drennan, Barry
• Assistant Examiner(s): NGUYEN, PHONG X
• Application Number:
  US12/811,268
• Publication Number:
  US 20100283781A1
• Time in Patent Office: 3,383 Days
• CPC Class Codes: A61C 13/0004 Computer-assisted sizing or machining of dental prostheses
• CPC Class Codes: G06T 17/00 Three dimensional [3D] modelling, e.g. data description of 3D objects
• CPC Class Codes: G06T 19/003 Navigation within 3D models or images
• CPC Class Codes: G06T 2207/10012 Stereo images
• CPC Class Codes: G06T 2210/41 Medical
• CPC Class Codes: G06V 20/64 Three-dimensional objects
• CPC Class Codes: G06V 20/653 by matching three-dimensional models, e.g. conformal mapping of Riemann surfaces
• CPC Class Codes: H04N 13/221 using the relative movement between cameras and objects

Abstract

A three-dimensional model of an object is employed to aid in navigation among a number of images of the object taken from various viewpoints. In general, an image of an object such as a digital photograph is displayed in a user interface or the like. When a user selects a point within the display that corresponds to a location on the surface of the object, another image may be identified that provides a better view of the object. In order to maintain user orientation to the subject matter while navigating to this destination viewpoint, the display may switch to a model view and a fly-over to the destination viewpoint may be animated using the model. When the destination viewpoint is reached, the display may return to an image view for further inspection, marking, or other manipulation by the user.

Claims

1. A method of navigating among a number of two-dimensional digital images taken of an object by a camera, the method comprising:

displaying a first image of the object from the camera, the first image selected from the number of images and showing a surface of the object from a first viewpoint; and

in response to receiving a selection of an image location within the first image, automatically performing the steps of;

determining a location on the surface of the object based on the selection of the image location;

selecting a second image of the object from the number of images taken of the object, the second image selected to provide an improved view of the location on the surface of the object from a second viewpoint, wherein the improved view is more normal to the surface of the object at the location than the first image;

transitioning from a first image view of the first image from the first viewpoint to a model view of a three-dimensional model of the object representing the object in three-dimensional space, wherein the three-dimensional model comprises a three-dimensional set of points forming a three-dimensional view of the object reconstructed from the number of images;

displaying an animation from the first viewpoint to the second viewpoint in the model view using the three-dimensional model, wherein the animation comprises displaying a spatial transition of the three-dimensional model from the first viewpoint to the second viewpoint;

transitioning from the model view to a second image view of the second image from the second viewpoint; and

displaying the second image,

except that if a user interrupt of the animation is received during display of the animation, then exiting the animation at a point of the interrupt in the model view with an automatic jump to a nearest image from the number of images at the point of the interrupt and displaying the nearest image in another image view.

2. The method of claim 1 wherein receiving the selection of the location includes receiving the selection from within a graphical user interface.

3. The method of claim 2 wherein receiving the selection includes an input from a cursor control tool.

4. The method of claim 1 wherein selecting the second image includes displaying a plurality of candidate images selected from the number of images taken of the object and receiving a user selection of one of the plurality of candidate images.

5. The method of claim 1 wherein the number of images includes a plurality of frames of video data that originate from a center channel of a video camera having one or more side channels that capture data used to generate the three-dimensional model.

6. The method of claim 1 wherein selecting the second image includes selecting a one of the number of images having the location nearest to a center thereof.

7. The method of claim 1 wherein the object includes human dentition.

8. The method of claim 7 wherein the object includes at least one of teeth, gums, dentures, and braces.

9. The method of claim 7 wherein the object includes a prepared tooth surface.

10. The method of claim 1 wherein the determining the location comprises projecting two-dimensional coordinates of the image location onto a surface of the three-dimension model to identify the location on the surface of the object.

11. The method of claim 1 wherein the displaying the animation comprises displaying a rotation of the three-dimensional model from the first viewpoint to the second viewpoint.

12. A system for navigating among a number of two-dimensional digital images taken of an object by a camera, the system comprising:

a computer;

a display operatively coupled to the computer;

a user input device that receives a user selection, the user input device operatively coupled to the computer; and

a computer-usable medium operatively coupled to the computer, the computer-usable medium having stored within it computer-readable instructions for execution by the computer to perform a method comprising the steps of;

displaying a first image of the object from the camera, the first image selected from the number of images and showing a surface of the object from a first viewpoint; and

in response to receiving a selection of an image location within the first image, automatically performing the steps of;

determining a location on the surface of the object based on the selection of the image location;

selecting a second image of the object from the number of images taken of the object, the second image selected to provide an improved view of the location on the surface of the object from a second viewpoint, wherein the improved view is more normal to the surface of the object at the location than the first image;

transitioning from a first image view of the first image from the first viewpoint to a model view of a three-dimensional model of the object representing the surface of the object in three-dimensional space, wherein the three-dimensional model comprises a three-dimensional set of points forming a three-dimensional view of the object reconstructed from the number of images;

displaying an animation from the first viewpoint to the second viewpoint in the model view using the three-dimensional model, wherein the animation comprises displaying a spatial transition of the three-dimensional model from the first viewpoint to the second viewpoint;

transitioning from the model view to a second image view of the second image from the second viewpoint; and

displaying the second image,

except that if a user interrupt of the animation is received during display of the animation, then exiting the animation at a point of the interrupt in the model view with an automatic jump to a nearest image from the number of images at the point of the interrupt and displaying the nearest image in another image view.

13. The system of claim 12 wherein the user input device includes a cursor control tool.

14. The system of claim 12 further comprising:

a three-dimensional camera having a left channel and a right channel that capture spatial information, the camera operatively coupled to the computer; and

computer-readable instructions for execution by the computer to construct the three-dimensional model from the spatial information, the computer-readable instructions stored within the computer-usable medium.

15. The system of claim 12 wherein the computer-usable medium includes one or more of a computer chip, an optical disk, and a magnetic disk.

16. The system of claim 12 wherein the display includes a three-dimensional display.

17. The system of claim 16 wherein the display includes an autostereoscopic display.

18. The system of claim 16 wherein receiving the selection of the location includes receiving the selection from within a graphical user interface.

19. The system of claim 16 wherein selecting the second image includes displaying a plurality of candidate images selected from the number of images taken of the object and receiving a user selection of one of the plurality of candidate images.

20. The system of claim 16 wherein selecting the second image includes selecting a one of the number of images having the location nearest to a center thereof.

21. The system of claim 12 wherein the determining the location comprises projecting two-dimensional coordinates of the image location onto a surface of the three-dimension model to identify the location on the surface of the object.

22. The system of claim 12 wherein the displaying the animation comprises displaying a rotation of the three-dimensional model from the first viewpoint to the second viewpoint.