COMPUTER-BASED METHOD AND SYSTEM OF DYNAMIC CATEGORY OBJECT RECOGNITION

• US 20140270361A1
• Filed: 03/15/2013
• Published: 09/18/2014
• Est. Priority Date: 03/15/2013
• Status: Active Grant

• Current Assignee: Toyota Jidosha Kabushiki Kaisha (Toyota Motor Corporation)
• Original Assignee: Toyota Jidosha Kabushiki Kaisha (Toyota Motor Corporation)
• Inventors: Amma, Ayako, Djugash, Joseph M.A.
• Granted Patent: US 9,111,348 B2
• Time in Patent Office: Days
• US Class Current: 382/103
• CPC Class Codes: G06T 2207/10028 Range image; Depth image; 3D point clouds
• CPC Class Codes: G06T 2207/20081 Training; Learning
• CPC Class Codes: G06T 7/75 involving models

Abstract

A computer-based method/system of dynamic category object recognition for estimating pose and/or positioning of target objects and target object'"'"'s parts. The method/system may recognize a target object and the target object'"'"'s parts. The method/system may segment and extract data corresponding to the target object and the target object'"'"'s parts, and estimate the pose and positioning of the target object and the target object'"'"'s parts using a plurality of stored object models. The dynamic method/system may supplement or modify the parameters of the plurality of stored object models and/or store learned object models. The learned object models assist in recognizing and estimating pose and/or positioning of newly encountered objects more accurately and with fewer processing steps. The method and system may include a processor, a sensor, an external device, a communications unit, and a database.

Claims

1. A computer-based method for estimating object pose and positioning information of a target object based on a visual data set corresponding to at least the target object, the method comprising the steps of:

providing, using a database, a plurality of object models, each having at least one parameter;

analyzing, using a processor, the visual data set in comparison to the plurality of object models;

fitting, using the processor and based on analyzing the visual data set in comparison to the plurality of object models, an object-fitting model onto, around, or within a portion of the analyzed visual data set corresponding to the target object, the object-fitting model having at least one parameter;

estimating, using the processor, the at least one parameter of the object-fitting model; and

storing, using the processor, a learned object model having at least one parameter in the database, based on the object-fitting model and the estimated at least one parameter of the object-fitting model.

2. The method of claim 1 further comprising:

recognizing, using the processor and by comparison of the visual data set to the plurality of object models, a target data set as being similar in shape or structure to the object-fitting model,

wherein the step of fitting the object-fitting model is based on the recognized target data set.

3. The method of claim 2 further comprising:

segmenting and extracting, using the processor, a cluster from the target data set, the cluster corresponding to the target object;

recognizing, using the processor, at least one of a plurality of cluster parts of the cluster as being similar in shape or structure to at least one corresponding part-fitting model of a plurality of part-fitting models; and

wherein the step of estimating the at least one parameter of the object-fitting model is based on analyzing, using the processor, the segmented and extracted cluster and analyzing, using the processor, the plurality of cluster parts.

4. The method of claim 2, further comprising:

recognizing, using the processor and by comparison of the visual data set to the plurality of object models, a first target data part of the target data set as being similar in shape or structure to a first part-fitting model having at least one parameter; and

estimating, using the processor, the at least one parameter of the first part-fitting model by analyzing, using the processor, the first target data part.

5. The method of claim 4, wherein the step of storing the learned object model is based on the estimated at least one parameter of the object-fitting model or the estimated at least one parameter of the first part-fitting model.

6. The method of claim 5 further comprising:

recognizing, using the processor and by comparison of the visual data set to the plurality of object models, a second target data part of the target data set as being similar at least in shape or structure to a second part-fitting model having at least one parameter; and

estimating, using the processor, the at least one parameter of the second part-fitting model by analyzing, using the processor, the second target data part.

7. The method of claim 6, wherein the step of storing the learned object model is based on the estimated at least one parameter of the object-fitting model, the estimated at least one parameter of the first part-fitting model, the estimated at least one parameter of the second part-fitting model, or combinations thereof.

8. The method of claim 7 further comprising:

segmenting and extracting, using the processor, a first cluster part from the cluster, the first cluster part corresponding to the first target data part, wherein the step of estimating the at least one parameter of the first part-fitting model is based on analyzing the segmented and extracted first cluster part; and

segmenting and extracting, using the processor, a second cluster part from the cluster, the second cluster part corresponding to the second target data part, wherein the step of estimating the at least one parameter of the second part-fitting model is based on analyzing the segmented and extracted second cluster part.

9. The method of claim 8 further comprising:

determining, using the processor, a first relationship between a positioning of the first part-fitting model and a positioning of the second part-fitting model;

determining, using the processor, a second relationship between a positioning of the first cluster part and a positioning of the second cluster part;

comparing, using the processor, the second relationship to the first relationship; and

estimating or modifying, using the processor, the positioning of the first part-fitting model and the positioning of the second part-fitting model based on the compared second relationship to the first relationship.

10. The method of claim 1 further comprising:

analyzing, using the processor, data corresponding to a newly encountered object in comparison to the stored plurality of object models and the learned object model;

recognizing, using the processor, a portion of the data corresponding to the newly encountered object as being similar in shape or structure to the learned object model;

fitting, using the processor, the learned object model onto, around, or within the cluster; and

estimating, using the processor, the at least one parameter of the learned object model based on the step of fitting the learned object model.

11. The method of claim 10 wherein the estimating the at least one parameter of the object-fitting learned model is based on modifying the at least one parameter of the object-fitting learned model to determine a close fit for the portion of the data corresponding to the newly encountered object.

12. The method of claim 10 further comprising:

outputting, using the processor and a communications unit coupled to the processor, an output data corresponding to an inquiry directed to eliciting a response regarding the newly encountered object.

13. A computer-based method for estimating pose and positioning information of target objects based on a visual data set including a first visual data and a second visual data, the method comprising the steps of:

providing, using a database, a plurality of object models, each having at least one parameter;

receiving or detecting, using a processor, the visual data set corresponding at least to a target object;

recognizing, using the processor and by comparison of the first visual data to the plurality of object models, a first target data adjacent to, around, or within a first enclosure of the first visual data as being similar in shape or structure to the first object model;

segmenting and extracting, using the processor, a cluster in the second visual data that corresponds to the target object based on the first enclosure;

analyzing, using the processor, the cluster in comparison to the stored plurality of object models;

fitting, using the processor, the learned object model onto, around, or within the analyzed cluster;

estimating, using the processor, the at least one parameter of the object-fitting model; and

storing, using the processor, a learned object model in the database, based on the object-fitting model and the estimated at least one parameter of the object-fitting model.

14. The method of claim 13, further comprising:

recognizing, using the processor and by comparison of the first visual data to the plurality of object models, a first target data part of the first target data adjacent to, around, or within a second enclosure as being similar in shape or structure to a first part-fitting model having at least one parameter;

segmenting and extracting, using the processor, a first cluster part of the cluster based on the second enclosure;

analyzing, using the processor, the first cluster part in comparison to the stored plurality of object models; and

fitting, using the processor, a first part-fitting model onto, around, or within the analyzed cluster, the first part-fitting model having at least one parameter.

15. The method of claim 14, wherein the step of storing the learned object model is based on the estimated at least one parameter of the object-fitting model or the estimated at least one parameter of the first part-fitting model.

16. The method of claim 15 further comprising:

recognizing, using the processor and by comparison of the first visual data to the plurality of object models, a second target data part of the first target data adjacent to, around, or within a third enclosure as being similar in shape or structure to a second part-fitting model having at least one parameter;

segmenting and extracting, using the processor, a second cluster part of the cluster based on the third enclosure;

analyzing, using the processor, the second cluster part in comparison to the stored plurality of object models; and

fitting, using the processor, a second part-fitting model onto, around, or within the analyzed second cluster part, the second part-fitting model having at least one parameter.

17. The method of claim 16, wherein the step of storing the learned object model is based on the estimated at least one parameter of the object-fitting model, the estimated at least one parameter of the first part-fitting model, the estimated at least one parameter of the second part-fitting model, or combinations thereof.

18. A computer-based method for estimating object pose and positioning information based on a visual data set corresponding to at least a target object, the method comprising the steps of:

providing, using a database, a plurality of object models, each having at least one parameter;

recognizing, using a processor and by comparison of the visual data set to the plurality of object models, a target data set adjacent to, around, or within a first enclosure as being similar in shape or structure to a first object model of the plurality of object models having at least one parameter;

analyzing, using the processor, the recognized target data set in comparison to the stored plurality of object models;

estimating, using the processor, the at least one parameter of the plurality of object models based on the analyzed visual data set in comparison to the stored plurality of object models; and

storing, using the processor, a learned object model in the database, based on the object-fitting model and the estimated at least one parameter of the object-fitting model.

19. The method of claim 18, further comprising:

segmenting and extracting, using the processor, a cluster adjacent to, around, or within a first enclosure of the target data set that corresponds to the target object; and

analyzing, using the processor, the cluster in comparison to the stored plurality of object models,

wherein the estimating the at least one parameter of the plurality of object models is based on the analyzed cluster.

20. The method of claim 19, further comprising:

fitting, using the processor and based on the estimated at least one parameter of the plurality of object models, an object-fitting model onto, around, or within the analyzed cluster, the object-fitting model having at least one parameter, wherein the stored learned object model is based on the object-fitting model; and

analyzing, using the processor, newly encountered objects using the plurality of stored object models and the learned object model.