JOINT SEMANTIC SEGMENTATION OF IMAGES AND SCAN DATA
First Claim
1. A method comprising:
- employing at least one processor executing computer executable instructions embodied on at least one computer readable medium to facilitate performing operations comprising;
receiving an image comprising image pixels comprising respective two-dimensional image features;
receiving image data points corresponding to the image, the image data points comprising respective three-dimensional image features;
correlating the image data points to corresponding ones of the image pixels associated with the image data points;
combining the respective two-dimensional image features and the respective three-dimensional image features of the image data points and the corresponding ones of the image pixels associated with the image data points and generating joint features for the image data points and the corresponding ones of the image pixels associated with the image data points; and
classifying the image pixels based on analyzing the joint features.
1 Assignment
0 Petitions
Accused Products
Abstract
Systems, methods, and apparatus are described that that increase computer vision analysis in the field of semantic segmentation. With images accompanied by scan data, both two-dimensional and three-dimensional image information is employed for joint segmentation. Through the established correspondence between image data and scan data, two-dimensional and three-dimensional information respectively associated therewith is integrated. Using trained random forest classifiers, the probability of each pixel in images belonging to different object classes is predicted. With the predicted probability, optimization of the labeling of images and scan data is performed by integrating multiples cues in the markov random field.
-
Citations
39 Claims
-
1. A method comprising:
employing at least one processor executing computer executable instructions embodied on at least one computer readable medium to facilitate performing operations comprising; receiving an image comprising image pixels comprising respective two-dimensional image features; receiving image data points corresponding to the image, the image data points comprising respective three-dimensional image features; correlating the image data points to corresponding ones of the image pixels associated with the image data points; combining the respective two-dimensional image features and the respective three-dimensional image features of the image data points and the corresponding ones of the image pixels associated with the image data points and generating joint features for the image data points and the corresponding ones of the image pixels associated with the image data points; and classifying the image pixels based on analyzing the joint features. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 10, 11, 12, 13, 14, 15, 16, 17)
-
9. The method of 8, wherein the predicting the probabilities comprises employing trained random forest classifiers.
-
18. A system, comprising:
-
a memory having computer executable components stored thereon; and a processor communicatively coupled to the memory, the processor configured to facilitate execution of the computer executable components, the computer executable components, comprising; an input component configured to receive an image comprising image pixels comprising respective two-dimensional image features, and to receive image data points corresponding to the image, the image data points comprising respective three-dimensional image features; a correlation component configured to correlate the image data points to corresponding ones of the image pixels associated with the image data points; an extraction component configured to combine the respective two-dimensional image features and the respective three-dimensional image features of the image data points and the corresponding ones of the image pixels associated with the image data points and form joint features for the image data points and the corresponding ones of the image pixels associated with the image data points; an analysis component configured to analyze the joint features; and a classification component configured to classify the image pixels based on the analysis. - View Dependent Claims (19, 20, 21, 22, 23, 24, 25, 27, 28, 30, 31, 32, 33)
-
-
26. The method of 25, wherein the analysis component is further configured to employ trained random forest classifiers to predict the probabilities.
-
29. The system of claim 29, wherein the analysis component is further configured to construct a markov random field wherein three-dimensional spatial smoothness constraints are encoded in a smooth term and employ the markov random field to determine a respective object class, from the respective object classes, to which the image data points belong.
-
34. A computer-readable storage medium comprising computer-readable instructions that, in response to execution, cause a computing system to perform operations, comprising:
-
receiving image data points corresponding to the image, the image data points comprising respective three-dimensional image features; correlating the image data points to corresponding ones of the image pixels associated with the image data points; combining the respective two-dimensional image features and the respective three-dimensional image features of the image data points and the corresponding ones of the image pixels associated with the image data points and generating joint features for the image data points and the corresponding ones of the image pixels associated with the image data points; and classifying the image pixels based on analyzing the joint features. - View Dependent Claims (35, 36, 37, 38, 39)
-
Specification