Methods, systems, and data structures for performing searches on three dimensional objects

• US 20040249809A1
• Filed: 01/23/2004
• Published: 12/09/2004
• Est. Priority Date: 01/25/2003
• Status: Active Grant

• Current Assignee: Purdue Research Foundation (The Trustees of Purdue University (d/b/a Purdue University))
• Original Assignee: Purdue Research Foundation (The Trustees of Purdue University (d/b/a Purdue University))
• Inventors: Iyer, Natraj, Jayanti, Subramaniam, Lou, Kuiyang, Ramani, Karthik
• Granted Patent: US 8,429,174 B2
• Time in Patent Office: Days
• US Class Current: 707/4
• CPC Class Codes: G06F 16/00 Information retrieval; Database structures therefor; File system structures therefor
• CPC Class Codes: G06F 16/248 Presentation of query results
• CPC Class Codes: G06F 16/5854 using shape and object relationship
• CPC Class Codes: G06F 2111/02 CAD in a network environment, e.g. collaborative CAD or distributed simulation
• CPC Class Codes: G06F 2111/20 Configuration CAD, e.g. designing by assembling or positioning modules selected from libraries of predesigned modules
• CPC Class Codes: G06F 2119/20 Design reuse, reusability analysis or reusability optimisation
• CPC Class Codes: G06F 30/10 Geometric CAD
• CPC Class Codes: G06F 30/27 using machine learning, e.g. artificial intelligence, neural networks, support vector machines [SVM] or training a model
• CPC Class Codes: G06V 10/34 Smoothing or thinning of the pattern; Morphological operations; Skeletonisation
• CPC Class Codes: G06V 20/64 Three-dimensional objects
• CPC Class Codes: G06V 30/1988 Graph matching graphical image representation G06V30/18181

Abstract

Techniques are provided for searching on three dimensional (3D) objects across large, distributed repositories of 3D models. 3D shapes are created for input to a search system; optionally user-defined similarity criterion is used, and search results are interactively navigated and feedback received for modifying the accuracy of the search results. Search input can also be given by picking 3D models from a cluster map or by providing the orthographic views for the 3D model. Feedback can be given by a searcher as to which models are similar and which are not. Various techniques adjust the search results according to the feedback given by the searcher and present the new search results to the searcher.

Claims

1. A method for searching, comprising:

receiving a three dimensional object;

searching one or more data stores with the three dimensional object as a first search query;

presenting results from the search, wherein the results include an answer set;

receiving modifications or relevance information for one or more items in the answer set; and

re-searching the one or more data stores with the modified or relevance information associated with the one or more items as a second search query.

2. The method of claim 1 further comprising, converting the three dimensional object into a graph skeleton defining a graph data structure, wherein the graph data structure is the first search query.

3. The method of claim 1 wherein receiving the three dimensional object further includes interactively permitting the three dimensional object to be sketched.

4. The method of claim 1 wherein the receiving the three dimensional object further includes presenting a list of three dimensional models and permitting the three dimensional object to be formed from selective ones of the list of three dimensional models.

5. The method of claim 1 wherein the presenting the results further include grouping selective portions of the one or more items in the answer set into related clusters.

6. The method of claim 1 further comprising, receiving one or more filters which constrain the first or second search queries.

7. The method of claim 1 wherein the re-searching further includes identifying in the relevance information for the one or more items information that identifies selective ones of the items that are more relevant to the first search query than selective other ones of the items.

8. A method of searching, comprising:

receiving a two dimensional object;

mapping the two dimensional object to a three dimensional representation;

searching one or more data stores with the three dimensional representation as a first search query; and

presenting one or more items in an answer set that is responsive to the first search query of the one or more data stores.

9. The method of claim 8 wherein the mapping further includes:

representing the two dimensional object as a two dimensional skeleton;

converting the two dimensional skeleton into a three dimensional skeleton; and

representing the three dimensional skeleton as a three dimensional graph structure, wherein the three dimensional graph structure is used as the first search query.

10. The method of claim 8 further comprising:

receiving relevance indications for a selective number of the one or more items in the answer set; and

searching the one or more data stores with the selective number of the one or more items and the relevance indications as a second search query.

11. The method of claim 10 retaining the relevance indications as preferences for subsequent search queries received and processed, where the retained relevance indications are used as filters to subsequent first queries.

12. The method of claim 8 further comprising organizing the answer set as a plurality of related clusters, wherein each related cluster includes a selective number of the one or more items.

13. The method of claim 8 wherein the mapping further includes:

converting the two dimensional object into a two dimensional skeleton;

generating candidate three dimensional vertices for each of two dimensions of the two dimensional skeleton;

generating candidate three dimensional edges from the candidate three dimensional vertices;

creating candidate three dimensional faces from the three dimensional edges on a same surface;

creating one or more three dimensional objects from the candidate three dimensional faces; and

associating the one or more three dimensional objects with the received two dimensional object as the three dimensional skeleton.

14. A method of presenting search results, comprising:

presenting three dimensional objects within one or more related clusters as an answer set in response to a first search query;

interactively receiving relevance information associated with selective ones of the three dimensional objects or selective ones of the related clusters; and

interactively issuing one or more second search queries based on the received relevance information to remove a number of the three dimensional objects from the answer set and to reduce a number of the related clusters from the answer set.

15. The method of claim 14 wherein the presenting further includes presenting the three dimensional objects within the one or more related clusters as answer set in response to the first query where the first query is a three dimensional search request derived from an initial two dimensional search request.

16. The method of claim 14 wherein the receiving of the relevance information further includes receiving selective feature vectors as a portion of the relevance information, wherein the selective feature vectors define selective ones of the three dimensional objects in more than one dimension.

17. The method of claim 16 wherein the receiving of the relevance information further includes defining the selective feature vectors based on lower level constructs associated with the selective ones of the three dimensional objects.

18. The method of claim 14 wherein the presenting further includes presenting the answer set in response to the first search query where the first search query is obtained by interactively traversing a hierarchy of available three dimensional models and selecting one of the available three dimension modes as the first search query.

19. The method of claim 14 wherein the presenting further includes forming the one or more related clusters using a Self Organizing Map (SOM) clustering technique.

20. The method of claim 19 wherein the presenting further includes using a SOM where feature vectors for each of the related clusters include a combination of a cluster feature vector for the three dimensional objects included in each cluster and graph distance measures obtained from comparisons of components associated with each of the three dimensional models included in each cluster.

21. A method to form a three dimensional search request, comprising:

deriving a topology associated with a three dimensional object;

deriving local geometries associated with components of the three dimensional object; and

forming the topology and the local geometries into a search request.

22. The method of claim 21 wherein the deriving the topology further includes obtaining relationships between geometric entities of the three dimensional object.

23. The method of claim 21 wherein the deriving the local geometries further includes obtaining local geometric characteristics for each geometric entity included within the three dimensional object.

24. A method of populating a searchable three dimensional data store, comprising:

reducing three dimensional models to three dimensional skeletons;

deriving for each three dimensional skeleton a topology and local geometries associated with each geometric entity defined in that skeleton;

creating a feature vector for each three dimensional skeleton and its associated topology and associated local geometries;

creating associations between each of the three dimensional skeletons and their associated feature vector; and

housing each association in a three dimensional data store.

25. The method of claim 24 wherein the deriving further includes:

representing the topologies as relationships between each of the geometric entities for the skeletons; and

representing the local geometries as local geometric characteristics for each of the geometric entities for the skeletons.

26. The method of claim 24 further comprising interfacing a high-level matcher with the three dimensional data store, wherein the high-level matcher evaluates topology information of the feature vectors in response to a search request issued to the three-dimensional data store.

27. The method of claim 26 further comprising interfacing a low-level matcher with the three dimensional data store, wherein the low-level matcher evaluates local geometry information of the feature vectors in response to the search request.

28. The method of claim 27 further comprising interfacing the high-level matcher with the low-level matcher to resolve candidate feature vectors which match the search request.

29. The method of claim 28 further comprising determining the candidate feature vectors which are responsive to the search request based on a substantial match between a search vector associated with the search request and the candidate feature vectors.

30. The method of claim 28 further comprising determining the candidate feature vectors which are responsive to the search request based on a predefined threshold percentage of a match between a search vector associated with the search request and the candidate feature vectors.

31. The method of claim 24 wherein the deriving further includes obtaining the local geometries as characteristics associated with geometric entities of the skeletons, wherein the characteristics includes at least one or more of entity volumes, entity volume distributions, and thickness distributions through distance.

32. A method for searching, comprising:

receiving a two or more dimensional object;

converting the two or more dimensional object into a three dimensional skeleton;

receiving a selection for a part of the three dimensional skeleton;

converting the part into a feature vector; and

searching a data store for the feature vector.

33. The method of claim 32 further comprising presenting candidate parts returned from the data store in response to the feature vector.

34. The method of claim 33 further comprising presenting candidate three dimensional models which have the candidate parts.

35. A method of searching, comprising:

receiving a two or more dimensional object;

reducing the two or more dimensional object to a three dimensional skeleton;

deriving from the two or more dimensional object text information;

forming a feature vector from topology and local geometries associated with the three dimensional skeleton and from the text information; and

submitting the feature vector as a search request to a data store.

36. The method of claim 35 wherein the deriving further includes associating the text information with at least one of a material identifier, a supplier identifier, a plant identifier, a designer identifier, a color identifier, a pattern identifier, and a date of creation identifier.

37. The method of claim 35 further comprising presenting an answer set associated with the search request that is clustered by related search results.

38. A method of performing a search, comprising:

acquiring a relevance preference for a three dimensional search request;

modifying the three dimensional search request with the relevance preference; and

submitting the three dimensional search request to one or more data stores.

39. The method of claim 38 further comprising:

presenting one or more clusters having one or more three dimensional models from an answer set returned from the one or more data stores in response to the three dimensional search request;

receiving additional relevance preferences for the answer set;

modifying the relevance preference with the additional relevance preferences; and

associating the modified relevance preference with a searcher that is associated with the search request.

40. The method of claim 39 further comprising:

receiving a new three dimensional search request from the searcher;

modifying the new three dimensional search request with the modified relevance preference associated with the searcher; and

submitting the new three dimensional search request to the one or more data stores.

41. A search system, comprising:

a three dimensional query interface;

a skeleton converter that interacts with the query interface; and

a feature vector constructor that interacts with the skeleton converter;

wherein the query interface receives search requests for objects in two or more dimensions and the objects are represented by the skeleton converter as three dimensional graphical data structures, the feature vector constructor translates the graphical data structures into feature vectors that are used as search requests to one or more data store, and wherein the feature vectors include topology and local geometric information derived from the three dimensional graphical data structures.

42. The search system of claim 41 further comprising:

a high-level matcher that interfaces to the one or more data stores which compares the topology information portions of the feature vectors against candidate feature vectors indexed in the one or more data stores; and

a low-level matcher that interfaces to the one or more data stores which compares the local geometric information portions of the feature vectors against the candidate feature vectors indexed in the one or more data stores.

43. The search system of claim 41 wherein the query interface presents answer sets associated with results of the search requests as one or more related clusters.

44. The search system of claim 43 wherein the query interface permits one or more search results included within the one or more clusters to be assigned relevance information.

45. The search system of claim 44 wherein the assigned relevance information is assigned automatically by the query interface or assigned via manual selections made by a searcher interfacing with the query interface.

46. The search system of claim 41 wherein the query interface receives the objects based on interactions with a searcher that interactive creates the objects within the query interface.

47. The search system of claim 41 wherein the query interface receives the objects based on interactive selections made by a searcher that traverses hierarchies of model objects.

48. The search system of claim 41 wherein the query interface receives the objects directly from a searcher via a paste operation or a file import operation.

49. The system of claim 41 wherein the query interface permits a searcher to interactively select parts associated with the graphical data structures, and wherein the feature vector constructor uses the parts to form the feature vectors.

50. A search system, comprising:

a three dimensional data store;

an indexer; and

a comparer;

wherein the indexer associates feature vectors with three dimensional graph data structures and the feature vectors include topology and local geometric information for entities of each graph data structure, and wherein the feature vectors and the graph data structures are associated with one another and housed in the data store, the comparer receives search requests as search vectors and searches the data store for candidate feature vectors that at least partially match the search vectors and returns the associated graph data structures as answer sets for the search vectors.

51. The search system of claim 50, wherein the comparer further includes:

a high-level matcher that matches topologies associated with the search vectors with the topology information for the candidate feature housed within the data store; and

a low-level matcher that matches local geometries associated with the search vectors with the local geometric information for the candidate feature vectors housed within the data store.

52. The search system of claim 50 wherein the comparer is configured to determine the at least partial match based on a preconfigured percentage of components of the search vectors that match candidate components of the candidate feature vectors.

53. The search system of claim 50 wherein the comparer is configured to determine the at least partial match based on a percentage of components of the search vectors that match candidate components of the candidate feature vectors, and wherein the percentage is part of the search vectors or included separately with the search vectors for consumption by the comparer.

54. The search system of claim 50 wherein the search vectors define parts of a three dimensional object.

55. The search system of claim 50 wherein the search vectors define complete three dimensional objects.

56. A query interface system, comprising:

a three dimensional search constructing interface;

a primitive data store of three dimensional primitives; and

a cluster interface;

wherein the search constructing interface is interacted with to assemble one or more of the three dimensional primitives from the primitive data store and form a three dimensional search query, and wherein the cluster interface organizes search results associated with results of search query into groupings of similar results.

57. The query interface of claim 56, wherein the cluster interface uses preconfigured or searcher defined criteria to organize the groupings.

58. The query interface of claim 56 further comprising a vector choice interface that permits a searcher to dynamically modify the search query by placing relevance information on one or more of the search results or placing the relevance information on portions of a single search result and resubmit the modified search query.

59. The query interface of claim 56 wherein the search constructing interface permits one or more new three dimensional primitives to be defined by a searcher and housed in the primitive data store.

60. The query interface of claim 56 wherein the search constructing interface permits a searcher to import or past a two or more dimensional object into the search constructing interface and reduces the object into derived primitives that are matched automatically to a selective number of the three dimensional primitives housed in the primitive data store.

61. A search constructor system, comprising:

a three dimensional candidate constructor; and

a feature vector constructor;

wherein the three dimensional candidate constructor receives two dimensional objects and converts them as three dimensional objects represented by three dimensional graph data structures, and wherein the feature vector constructor derives topology and local geometric information from the three dimensional graph structures to construct search vectors for a three dimensional search.

62. The search constructor system of claim 61, wherein the three dimensional candidate constructor generates three dimensional candidate vertices from vertices included in the two dimensional objects, generates three dimensional candidate edges from the three dimensional candidate vertices, constructs three dimensional candidate faces from the three dimensional candidate edges, and constructs the three dimensional objects from the three dimensional candidate faces.

63. The search constructor system of claim 62, wherein the three dimensional candidate constructor matches loops and the three dimensional candidate vertices to form geometric entities.

64. A geometric graph data structure, implemented in a computer readable medium, the graph data structure used to define three dimensional objects for purposes of search and retrieval, the graph data structure comprising:

one or more geometric entities;

a topology matrix that defines topology connections between the one or more geometric entities; and

one or more geometric shapes having geometric features associated with one or more geometric entities and wherein each geometric feature can be reduced to a single one of the geometric entities;

wherein the geometric graph data structure is generated from an object having two or more dimensions and wherein a feature vector is derived for the geometric data structure, wherein the feature vector that includes portions of the topology matrix and the features.

65. The graph data structure of claim 64, wherein each of the entities are defined by a set of edges, a set of loops, a set of surfaces, and a set of holes.

66. The graph data structure of claim 64, wherein each of the one or more shapes further include a shape topology matrix that defines topological connectivity between the geometric features of each shape.

67. A feature vector implemented in a computer readable medium, the feature vector defining searchable attributes of a three dimensional object and used for generating search requests or for populating a searchable data store with as search keys associated with the three dimensional objects, the feature vector comprising:

topology information that defines the connectivity and relationships between geometric entities associated with a three dimensional object; and

local geometries associated with characteristics of each of the geometric entities.

68. The feature vector of claim 67, wherein the feature vector is associated with a single part of the three dimensional object.

69. The feature vector of claim 67, wherein the feature vector is associated with one or more parts of the three dimensional object but not the complete three dimensional object.

70. The feature vector of claim 67, wherein the feature vector is used to perform a search against a three dimensional data store having a plurality of instances of the feature vector, wherein each of the instances represents a different three dimensional object instance or a different part of the different three dimensional object instances.

71. The feature vector of claim 67, wherein the feature vector is used to determine a partial match against one or more of the plurality of instances of the feature vector housed in the three dimensional data store.

72. A method of searching, comprising:

selecting features associated with an object represented in more than two dimensions;

selectively weighting the features; and

submitting the selectively weighted features as a search request.

73. The method of claim 72 wherein selecting the features further includes selecting between graph data structure elements representing the object or selecting portions of a feature vector describing the object.

74. The method of claim 72 wherein selectively weighting further includes weighting local or global features associated with the object.