Structured Knowledge Modeling and Extraction from Images
First Claim
1. In a digital medium environment to learn a model that is usable to explicitly correlate image features of an input image with text features automatically and without user intervention, a method implemented by at least one computing device comprising:
- obtaining training data by the at least one computing device, the training data including images and associated text;
extracting text features as structured semantic knowledge from the associated text of the training data using natural language processing by the at least one computing device;
training a model using the structured semantic knowledge and the images as part of machine learning by the at least one computing device, the model once trained is configured to form a structured image representation to explicitly correlate the image features of the input image with at least one of the extracted text features.
2 Assignments
0 Petitions
Accused Products
Abstract
Techniques and systems are described to model and extract knowledge from images. A digital medium environment is configured to learn and use a model to compute a descriptive summarization of an input image automatically and without user intervention. Training data is obtained to train a model using machine learning in order to generate a structured image representation that serves as the descriptive summarization of an input image. The images and associated text are processed to extract structured semantic knowledge from the text, which is then associated with the images. The structured semantic knowledge is processed along with corresponding images to train a model using machine learning such that the model describes a relationship between text features within the structured semantic knowledge. Once the model is learned, the model is usable to process input images to generate a structured image representation of the image.
-
Citations
20 Claims
-
1. In a digital medium environment to learn a model that is usable to explicitly correlate image features of an input image with text features automatically and without user intervention, a method implemented by at least one computing device comprising:
-
obtaining training data by the at least one computing device, the training data including images and associated text; extracting text features as structured semantic knowledge from the associated text of the training data using natural language processing by the at least one computing device; training a model using the structured semantic knowledge and the images as part of machine learning by the at least one computing device, the model once trained is configured to form a structured image representation to explicitly correlate the image features of the input image with at least one of the extracted text features. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 9, 10, 11)
-
-
12. In a digital medium environment to use a model to explicitly correlate text features with image features of an input image automatically and without user intervention, a system by at least one computing device comprising:
-
an extractor module to extract structured semantic knowledge from text associated with images in training data using natural language processing, the structured semantic knowledge describing the text features; a model training module to train a model using the structured semantic knowledge and image features of the images in the training data as part of machine learning by the at least one computing device, the model configured for determining probabilities to predict how well image features of the input image correlate to the extracted text features. - View Dependent Claims (13, 14, 15, 16, 17, 18)
-
-
19. In a digital medium environment to use a model as part of an image search to locate an input image automatically and without user intervention, a method implemented by at least one computing device comprising:
-
extracting structured semantic knowledge from text associated with images in training data using natural language processing by the at least one computing device, the structured semantic knowledge describing text features; training a model using the structured semantic knowledge and the images from the training data as part of machine learning by the at least one computing device; forming a structured image representation of the input image, by the at least one computing device, that explicitly correlates at least part of the text features of the training data with image features of the input image; and locating the input image as part of an image search by the at least one computing device, using the structured image representation. - View Dependent Claims (20)
-
Specification