Translating user input through two-dimensional images into three-dimensional scene

• US 20050243084A1
• Filed: 05/03/2004
• Published: 11/03/2005
• Est. Priority Date: 05/03/2004
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Schechter, Greg D., Lehenbauer, Daniel R., Smith, Adam M.
• Granted Patent: US 8,031,190 B2
• Time in Patent Office: Days
• US Class Current: 345/419
• CPC Class Codes: G06T 15/00 3D [Three Dimensional] image rendering
• CPC Class Codes: G06T 17/005 Tree description, e.g. octree, quadtree

Abstract

A computing system translates two-dimensional (2D) graphical input by a user who is selecting one or more 2D images in a three-dimensional (3D) scene of 3D models created by model 3D objects. The computing system comprises a viewport module, a retrieve module, set ray module and, a hit detection module. The viewport module defines 2D boundaries of the 3D scene and a view point location in 3D space for viewing the 3D scene. The retrieve module retrievies a selection point location in 2D space for the 2D graphical input. The set ray module sets a pick ray in 3D space based on the view point location and the selection point location. The hit detection module detects a hit by the pick ray on a 3D model in the 3D scene. In this computing system there is a method of processing a hierarchy of computer program visual objects for detecting a hit by 2D input on 2D and 3D images displayed by a computing system. The method begins by traversing branches of a first tree hierarchy of visual objects to leaf objects. Next the method detects whether the next unprocessed leaf object is a visual 2D object with 2D geometry or a model 3D object with 3D geometry. Lastly the method calls a 2D process to detect a hit on the 2D geometry of a visual object if a visual 2D object is detected and calls the 3D process to detect a hit on the 3D geometry of a visual object if a model 3D object is detected.

Claims

1. A computer data structure for translating user input applied to computer program objects for creating three-dimensional (3D) images and two-dimensional (2D) images in a visual tree object hierarchy having visual 2D objects and model 3D objects, the data structure comprising:

an object tree hierarchy;

one or more visual 2D objects in the tree hierarchy defining operations drawing a 2D image with a 2D geometry;

one or more model 3D objects in the tree hierarchy defining operations drawing a 3D image with a 3D geometry;

one or more viewport 3D objects defining operations converting a 2D input to a 3D scene made up of the one or more 3D models to a ray penetrating into the 3D scene, the viewport 3D object containing camera data; and

the camera data defining a camera view point location in 3D space from which to view the 3D scene.

2. The data structure of claim 1 wherein the 2D input is a cursor point location in 2D space.

3. The data structure of claim 2 wherein the camera view point location and cursor point location define the direction of the ray penetrating into the 3D scene.

4. A method for processing a hierarchy of computer program visual objects for detecting a hit by 2D input on two dimensional (2D) and three-dimensional (3D) images displayed by a computing system, the method comprising:

traversing branches of a first tree hierarchy to reach 2D and 3D visual objects;

detecting whether the next unprocessed object is a visual 2D object with 2D geometry or a model 3D object with 3D geometry; and

calling a 2D process to detect a hit on the 2D geometry of a visual object if a visual 2D object is detected and calling the 3D process to detect a hit on the 3D geometry of a visual object if a model 3D object is detected.

5. The method of claim 4, the 3D process comprises:

setting a camera view point; and

converting the 2D input into a ray based on the camera view point, the ray penetrating the 3D space of the images of one or more 3D models defined by the model 3D objects.

6. The method of claim 5, wherein the 2D input is a point in 2D space identified by a user-controlled pointing device.

7. The method of claim 6, wherein the 2D input is a cursor point location.

8. The method of claim 5 wherein a tree hierarchy defining a model 3D object has at least one content visual 2D object defining 2D content for a 3D model of the model 3D object and the method further comprises:

detecting the 2D content visual object for the 3D model;

converting the input from a ray to a cursor point location.

9. The method of claim 8 further comprises:

comparing the cursor point location to the 2D geometry of the content visual 2D object;

detecting a hit on the content visual 2D object when the cursor point location is within the 2D space of the content visual object.

10. The method of claim 5 further comprises:

comparing the ray to the 3D geometry of each model 3D object;

detecting a hit on the model 3D object when the ray intersects a 3D model drawn by the model 3D object.

11. A computing system for translating two-dimensional (2D) graphical input by a user selecting one or more 2D images in a three-dimensional (3D) scene of 3D models created by model 3D objects, said computing system comprising:

a viewport module defining 2D boundaries of the 3D scene and a view point location in 3D space for viewing the 3D scene;

a retrieve module retrieving a selection point location in 2D space for the 2D graphical input;

a set module setting a pick ray in 3D space based on the view point location and the selection point location; and

a hit detection module detecting a hit by the pick ray on a 3D model in the 3D scene.

12. The computing system of claim 11 wherein the 2D graphical input is a cursor point on a computer display screen.

13. The computing system of claim 12 further comprises:

a report module reporting the 3D hit results of the hit on the 3D model.

14. The computing system of claim 13 wherein the 3D hit results include the model hit, the intersection point, and the distance of the intersection point along the pick ray.

15. The computing system of claim 12 further comprises:

a 2D content detect module detecting 2D content in the image of a 3D model, the 2D content created by a content visual 2D object;

a 2D conversion module converting the pick ray to a cursor location; and

a 2D hit detect module detecting a hit on the 2D content based on the cursor location and 2D geometry defined by the content visual 2D object.