SYSTEM AND METHOD FOR MULTIRESOLUTION AND MULTITEMPORAL IMAGE SEARCH

• US 20170235767A1
• Filed: 12/27/2016
• Published: 08/17/2017
• Est. Priority Date: 06/05/2013
• Status: Active Grant

• Current Assignee: Maxar Intelligence Incorporated (Galileo Parent, Inc.)
• Original Assignee: DigitalGlobe, Inc. (Galileo Parent, Inc.)
• Inventors: Tusk, Carsten
• Granted Patent: US 10,482,122 B2
• Time in Patent Office: Days
• CPC Class Codes: G06F 16/51 Indexing; Data structures therefor; Storage structures
• CPC Class Codes: G06F 16/56 having vectorial format
• CPC Class Codes: G06F 16/583 using metadata automatically derived from the content
• CPC Class Codes: G06F 16/5838 using colour
• CPC Class Codes: G06F 16/5854 using shape and object relationship
• CPC Class Codes: G06F 16/5862 using texture
• CPC Class Codes: G06F 16/9535 Search customisation based on user profiles and personalisation
• CPC Class Codes: G06T 2207/20021 Dividing image into blocks, subimages or windows
• CPC Class Codes: G06T 7/11 Region-based segmentation

Abstract

A system for modular image mining and search, comprising a plurality of image capture devices, a search engine, an indexing engine, a database, and user interface software. A plurality of image capture devices capture images and store them to a database. The indexing engine indexes images to create image data for each image with associated metadata and stores the information to the database. The interface software is used to request search queries via a search engine. The search engine accesses the image data to find images or parts of images that satisfy the search query.

Claims

1. A system for multiresolution and multitemporal image mining and search, the system comprising:

an indexing engine software module operating on a first network-attached server computer and configured to index image data captured by a plurality of image capture devices by performing the following steps;

for each of a plurality of spatial scales, dividing each image of the captured image data into a plurality of tiles of the corresponding spatial scale, thereby generating a set of tiles of varying spatial scales for each image;

using a plurality of feature extractors, automatically extracting a plurality of features for each tile;

generating a vector for each tile, the vector comprising a numeric value for each of the plurality of features corresponding to a degree of presence of each corresponding feature; and

composing the set of generated vectors into an inverted feature presence index;

an image information database operating on the first network-attached server computer and adapted to receive and store image data and indices from the indexing engine software module; and

a search engine software module operating on a second network-attached server computer and configured to receive text-based search requests from users via the interface software module for searches of elements and features within an image;

wherein the indexing and categorization of image data is performed using automated feature extraction at multiple spatial scales, including at least pixel level features, scene level features including object identities, locations and counts, and region level features, and at multiple times.

2. A method for multiresolution and multitemporal searching of image data, the method comprising the steps of:

(a) receiving image data captured by a plurality of image capture devices;

(b) for each of a plurality of spatial scales, dividing each image of the received image data into a plurality of tiles of the corresponding spatial scale;

(c) using a plurality of feature extractors, automatically extracting a plurality of features for each tile;

(d) generating a vector for each tile, the vector comprising a numeric value for each of the plurality of features corresponding to a degree of presence of each corresponding feature; and

(e) composing the set of generated vectors into an inverted feature presence index;

(f) storing the image data and the inverted feature presence index in a database; and

(g) searching the image data and the image indices using a search engine stored in a memory of and operating on a processor of a second computing device using input received from a user via a network.

3. A system for searching over multiple spatial resolutions on a multi-temporal grid of images, the system comprising:

an image storage repository operating on a network-attached server computer and storing a plurality of images each stored at a plurality of spatial resolutions, the plurality of spatial resolutions of each of the plurality of images decomposed from the respective image as captured by a plurality of image capture devices that capture the plurality of images over a period of time and store them in the image storage repository with timestamps;

an indexing engine software module operating on a first network-attached server computer and configured to analyze the plurality of images is analyzed by;

performing automated feature extraction at multiple spatial scales, including at least pixel level features, scene level features including object identities, locations and counts, and region level features, and at multiple times; and

indexing image data captured by the plurality of image capture devices by performing the following steps;

for each of a plurality of spatial scales, dividing each image of the captured image data into a plurality of tiles of the corresponding spatial scale, thereby generating a set of tiles of varying spatial scales for each image;

using a plurality of feature extractors, automatically extracting a plurality of features for each tile;

generating a vector for each tile, the vector comprising a numeric value for each of the plurality of features corresponding to a degree of presence of each corresponding feature; and

composing the set of generated vectors into an inverted feature presence index; and

a search engine software module stored and operating on a network-attached server computer and configured to;

select an appropriate image resolution from a set of resolutions; and

conduct a search of a set of images at the selected image resultion to identify changes within a scene over a period of time in a set of images captured by a plurality of image capture devices, using at least the inverted feature presence index.