SYSTEMS AND METHODS FOR MANAGING LARGE VOLUMES OF DATA IN A DIGITAL EARTH ENVIRONMENT

• US 20140280158A1
• Filed: 03/13/2014
• Published: 09/18/2014
• Est. Priority Date: 03/15/2013
• Status: Active Grant

• Current Assignee: 12995514 Canada Incorporated
• Original Assignee: PYXIS innovation Inc.
• Inventors: Shatz, Idan
• Granted Patent: US 9,600,538 B2
• Time in Patent Office: Days
• US Class Current: 707/737
• CPC Class Codes: G06F 16/2462 Approximate or statistical queries

Abstract

A computer-implemented method for managing large volumes of data comprises dividing data about a number of features into a plurality of data groups, each of the groups having a plurality of features, each of the features having a plurality of properties, and each of the properties having a property value; for each of the groups, determining a number of distribution ranges for the property values for each of the properties; for each of the groups, determining a number of features having property values that are within each of the distribution ranges; and generating a summary associated with each of the groups, the summary comprising the properties of the features in the group and the number of the features that are within each of the distribution ranges for the properties.

Claims

1. A computer-implemented method for managing large volumes of data comprising:

(a) dividing data about a number of features into a plurality of data groups, each of the groups having a plurality of features, each of the features having a plurality of properties, and each of the properties having a property value;

(b) for each of the groups, determining a number of distribution ranges for the property values for each of the properties;

(c) for each of the groups, determining a number of features having property values that are within each of the distribution ranges; and

(d) generating a summary associated with each of the groups, the summary comprising the properties of the features in the group and the number of the features that are within each of the distribution ranges for the properties.

2. The method of claim 1, further comprising:

(a) determining the number of features in each group;

(b) if the number of features in each group is outside a group-size range, subdividing that group to generate a plurality of additional data groups that have less number of the features per group;

(c) repeating steps (a) and (b) until each of the number of features in each data groups is within the group size.

3. The method of claim 1, further comprising:

(a) repeatedly aggregating a number of summaries to generate higher-level summaries associated with a plurality of groups of features; and

(b) repeatedly aggregating a number of higher-level summaries until an overall summary associated with all the groups is obtained.

6. The method of claim 2, further comprising repeating the steps of claim 5 until a preliminary response within a acceptable margin of error is obtained.

4. A computer-implemented method for processing queries comprising:

(a) receiving a query;

(b) receiving at least one summary associated with one or more data groups, each of the groups having a plurality of features, each of the features having a plurality of properties, each of the properties having a property value, and the at least one summary including the properties of the features in the group and the number of the features that are within each of the distribution ranges for the properties;

(c) conducting the query using the at least one summary to obtain a preliminary response to the query; and

(d) generating a margin of error associated with the preliminary response based upon the distribution ranges for the properties in the at least one summary.

5. The method of claim 4, further comprising:

(a) receiving at least one additional lower-level summary that is of a different level than a previously received summary;

(b) conducting the query using the at least one lower-level summary to obtain a second preliminary response to the query; and

(c) generating a margin of error associated with the second preliminary response based upon the margin of error associated with the at least one summary.

7. The method of claim 5, further comprising determining which lower-level summary to receive based upon the at least one previously summary.

8. An information system comprising:

(a) at least one server having access to a hierarchical levels of summaries associated with one or more data groups, each of the groups having a plurality of features, each of the features having a plurality of properties, each of the properties having a property value, and the at least one summary including the properties of the features in the group and the number of the features that are within each of the distribution ranges for the properties;

(a) at least one electronic device in data communication with the server, the at least one electronic device configured to;

(i) receive a query,

(ii) receive at least one summary from the at least one server,

(iii) conducting the query using the at least one summary to obtain a preliminary response to the query, and

(iv) generating a margin of error associated with the preliminary response based upon the margin of error associated with the at least one summary.

9. The system of claim 8, wherein the data groups are aligned on a common geospatial grid.

10. The system of claim 9, wherein the summaries can be divided and combined for transmission thereof.

11. The system of claim 8, wherein the at least one electronic device comprises a plurality of devices in data communication with each other via a peer to peer network.