Computer vision CAD model

• US 8,780,110 B2
• Filed: 03/18/2013
• Issued: 07/15/2014
• Est. Priority Date: 10/12/2007
• Status: Active Grant

• Current Assignee: MvTec Software GmbH (MvTec LLC)
• Original Assignee: MvTec Software GmbH (MvTec LLC)
• Inventors: Ben Himane, Selim, Hinterstoisser, Stefan, Navab, Nassir
• Primary Examiner(s): Sajous, Wesner
• Assistant Examiner(s): Shin, Andrew
• Application Number:
  US13/845,946
• Publication Number:
  US 20130226532A1
• Time in Patent Office: 484 Days
• Field of Search: 345/419, 345/581, 345/582, 345/619, 382/103, 382155-160, 382/181, 382/190, 382/195, 382/199, 382/201, 382/203, 382/205, 382/209, 382224-228, 700 28- 31, 700/47, 700/48, 706 12- 16, 706/20, 706/21
• US Class Current: 345/419
• CPC Class Codes: G06F 18/254 of classification results, e.g. of results related to same input data
• CPC Class Codes: G06F 30/00 Computer-aided design [CAD]
• CPC Class Codes: G06T 7/75 involving models
• CPC Class Codes: G06V 10/40 Extraction of image or video features
• CPC Class Codes: G06V 10/809 of classification results, e.g. where the classifiers operate on the same input data

Abstract

The CV-CAD (computer vision-computer-aided design) model is an enhanced CAD (computer-aided design) model that integrates local and global computer vision data in order to represent an object not only geometrically but also in terms of computer vision. The CV-CAD model provides a scalable solution for intelligent and automatic object recognition, tracking and augmentation based on generic models of objects.

Claims

1. A method for creating an enhanced CAD, referred to as CV-CAD model, called Computer Vision CAD model wherein the CV-CAD model describes an object consisting of one or more object parts, each object part comprising (i) a standard CAD model, said CAD model comprising information about geometry and, optionally, texture of said object part and (ii) at least one agent, said at least one agent being associated to said object part, said at least one agent comprising necessary information that allows

(a) to generate an output given an input digital image by considering only said object part, wherein the generated output is the result of the recognition or the detection or the estimation of the pose of said object part, and

(b) to provide a confidence value of said generated output, and

(c) to generate the said output independently from any other agent'"'"'s output, wherein said method comprises the steps of:

(a1) training for each said object part at least one said agent based on the geometry and, optionally, the surface texture corresponding to said object part to which said agent is associated,

(b1) training geometric neighborhood relations and visibility information between said at least one agent, and

(b2) creating the CV-CAD model by storing said CAD models and said corresponding at least one agent of said one or more object parts in one or more digital files, wherein each said agent consists of an executable code or the result of said training or a set of control parameters.

2. The method of claim 1, wherein training said at least one agent for said object part comprises selecting the best fitting at least one computer vision method and the corresponding best set of computer vision features within said object part to enable said output and confidence value computation from said object part.

3. The method of claim 1, wherein said at least one agent in said object part is removable, addable, or replaceable by at least one other agent if said object part is removed, added, or replaced.

4. The method of claim 1, wherein said at least one agent in said object part is selected from any of the following agents:

point feature agent, edge-based agent, contour-based agent, color-based agent.

5. A method for using an enhanced CAD, referred to as CV-CAD model, called Computer Vision CAD model, wherein the CV-CAD model describes an object consisting of one or more object parts, each object part comprising (I) a standard CAD model, said CAD model comprising information about geometry and, optionally, texture of said object part and (II) a plurality of agents, said plurality of agents being associated to said object part, said plurality of agents comprising necessary information that allows

(a) to generate an output given an input digital image by considering only said object part, wherein the generated output is the result of the recognition or the detection or the estimation of the pose of said object part, and

(b) to provide a confidence value of said generated output, and

(c) to generate the said output independently from any other agent'"'"'s output, and

(d) to provide information about the geometric neighborhood relations and visibility information between said plurality of agents,

wherein the said method comprises the steps of:

(e) receiving said CV-CAD model,

(f) generating one or more executable instances of said plurality of agents for each said part of the object that are stored in said CV-CAD model,

(g) receiving an image,

(h) computing the outputs of said object parts described in said CV-CAD model by executing for each said object part its associated said plurality of agents, each of said plurality of agents generates the said output based on said object part independently from any other agent'"'"'s output,

(i) computing for each said output said confidence value from said object part and image region.

6. The method of claim 5, wherein said at least one agent in said object part is removable, addable, or replaceable by at least one other agent if said object part is removed, added, or replaced.

7. The method of claim 5, wherein the communication between all said agents leads to the consensus on said output of said object or said object parts and to the conclusion on how said object or said object parts are currently geometrically related to the camera or to other said object parts, leading to an indication of visibility derived from the computed said output and the geometry of said object or said object parts.

8. The method of claim 5, wherein said at least one agent selects the best set of computer vision features within the said local region and the best corresponding computer vision methods to enable said output and confidence value computation from said object part.

9. A system for creating an enhanced CAD, referred to as CV-CAD model, called Computer Vision CAD model, wherein the CV-CAD model describes an object consisting of one or more object parts, each object part comprising (I) a standard CAD model, said CAD model comprising information about geometry and, optionally, texture of said object part and (II) at least one agent, said at least one agent being associated to said object part, said at least one agent comprising necessary information that allows

(a) to generate an output given an input digital image by considering only said object part, wherein the generated output is the result of the recognition or the detection or the estimation of the pose of said object part, and

(b) to provide a confidence value of said generated output, and

(c) to generate the said output independently from any other agent'"'"'s output,

wherein the said system comprises:

(d) means for training for each said object part at least one said agent based on the local geometry and, optionally, the local surface texture corresponding to the local region of said object part to which said agent is associated,

(e) means for training geometric neighborhood relations and visibility information between said at least one agent, and

(f) means for creating the CV-CAD model by storing said CAD models and said corresponding at least one agent of said one or more object parts in one or more digital files, wherein each said agent consists of an executable code or the result of said training or a set of control parameters.

10. A system for using an enhanced CAD, referred to as CV-CAD model, called Computer Vision CAD model, wherein the CV-CAD model describes an object consisting of one or more object parts, each object part comprising (I) a standard CAD model, said CAD model comprising information about geometry and, optionally, texture of said object part and (II) at least one agent, said at least one agent being associated to said object part, said at least one agent comprising necessary information that allows

(a) to generate an output given an input digital image by considering only said object part, wherein the generated output is the result of the recognition or the detection or the estimation of the pose of said object part, and

(b) to provide a confidence value of said generated output, and

(c) to generate the said output independently from any other agent'"'"'s output, and

(d) to provide information about the geometric neighborhood relations and visibility information between said at least one agent, wherein the said system comprises:

(e) means for receiving said CV-CAD model,

(f) means for generating one or more executable instances of said at least one agent for each said part of the object that are stored in said CV-CAD model,

(g) means for receiving an image,

(h) means for computing the outputs of said object parts described in said CV-CAD model by executing for each said object part its associated said at least one agent, each of said at least one agent generates the said output based on said object part independently from any other agent'"'"'s output

(i) means for computing for each said output said confidence value from said object part and image region.