Navigating among images of an object in 3D space

• US 10,503,962 B2
• Filed: 03/02/2018
• Issued: 12/10/2019
• Est. Priority Date: 01/04/2008
• Status: Active Grant

• Current Assignee: Medit Corp.
• Original Assignee: Midmark Corporation
• Inventors: Kriveshko, Ilya A., Dedkov, Dmitriy A.
• Primary Examiner(s): Nguyen, Phong X
• Application Number:
  US15/910,124
• Publication Number:
  US 20180196995A1
• Time in Patent Office: 648 Days
• CPC Class Codes: A61C 13/0004 Computer-assisted sizing or machining of dental prostheses
• CPC Class Codes: G06T 17/00 Three dimensional [3D] modelling, e.g. data description of 3D objects
• CPC Class Codes: G06T 19/003 Navigation within 3D models or images
• CPC Class Codes: G06T 2207/10012 Stereo images
• CPC Class Codes: G06T 2210/41 Medical
• CPC Class Codes: G06V 20/64 Three-dimensional objects
• CPC Class Codes: G06V 20/653 by matching three-dimensional models, e.g. conformal mapping of Riemann surfaces
• CPC Class Codes: H04N 13/221 using the relative movement between cameras and objects

Abstract

A three-dimensional model of an object is employed to aid in navigation among a number of images of the object taken from various viewpoints. In general, an image of an object such as a digital photograph is displayed in a user interface or the like. When a user selects a point within the display that corresponds to a location on the surface of the object, another image may be identified that provides a better view of the object. In order to maintain user orientation to the subject matter while navigating to this destination viewpoint, the display may switch to a model view and a fly-over to the destination viewpoint may be animated using the model. When the destination viewpoint is reached, the display may return to an image view for further inspection, marking, or other manipulation by the user.

Claims

1. A computing device comprising a display screen, the computing device configured to display on the display screen a graphical user interface comprising an image display area that alternately displays:

a first image of an object, the first image selected from a number of images taken of the object, and showing a surface of the object from a first viewpoint;

an animation of a spatial transition from the first viewpoint to a second viewpoint that provides an improved view of a user-selected location on the surface of the object in the first image, the animation rendered from a three-dimensional model of the object; and

a second image of the object selected from the number of images and showing the object from the second viewpoint,

except that if a user interrupt of the animation is received during the display of the animation, then the image display area automatically exits the animation at a point of the interrupt and displays a nearest image from the number of images at the point of the interrupt.

2. The computing device of claim 1 wherein the animation is a texture-mapped animation.

3. The computing device of claim 1 wherein the first image and the second image are selected from a plurality of frames of video data that originate from a center channel of a video camera having one or more side channels that capture data used to generate the three-dimensional model.

4. The computing device of claim 3 wherein the user interface displays a two-dimensional map illustrating a current location of the camera.

5. The computing device of claim 4 wherein the two-dimensional map is displayed in the image display area.

6. The computing device of claim 3 wherein the user interface displays a three-dimensional map illustrating a current location of the camera.

7. The computing device of claim 6 wherein the three-dimensional map is displayed in the image display area.

8. The computing device of claim 1 wherein the object includes human dentition.

9. The computing device of claim 8 wherein the object includes at least one of teeth, gums, dentures, and braces.

10. The computing device of claim 8 wherein the object includes a prepared tooth surface.

11. The computing device of claim 1 wherein the user interface displays controls for receiving the user interrupt.

12. The computing device of claim 1 wherein the images are color two-dimensional images.

13. The computing device of claim 1 wherein the images are grayscale two-dimensional images.

14. The computing device of claim 1 wherein the user interface displays position and orientation information during the animation.

15. The computing device of claim 1 wherein the user interface displays a label, a landmark, or a grid during the animation.

16. The computing device of claim 1 wherein the image display area displays the animation with an acceleration from the first viewpoint and a deceleration to the second viewpoint.

17. The computing device of claim 1 wherein the image display area displays the animation with a zoom out at or near the first viewpoint.