Accessing data held in large databases

• US 5,675,786 A
• Filed: 01/30/1995
• Issued: 10/07/1997
• Est. Priority Date: 01/31/1994
• Status: Expired due to Term

• Current Assignee: Neil Howard Mckee, Peter Phaal
• Original Assignee: Neil Howard Mckee, Peter Phaal
• Inventors: Phaal, Peter, McKee, Neil Howard
• Primary Examiner(s): Black, Thomas G.
• Assistant Examiner(s): LEWIS, CHERYL RENEA
• Application Number:
  US08/380,908
• Time in Patent Office: 981 Days
• Field of Search: 395/600, 395/650, 364/406
• US Class Current: 1/1
• CPC Class Codes: G06F 16/2462 Approximate or statistical queries
• CPC Class Codes: G06Q 40/128 Check-book balancing, updating or printing arrangements
• CPC Class Codes: Y10S 707/99933 Query processing, i.e. searching
• CPC Class Codes: Y10S 707/99934 Query formulation, input preparation, or translation
• CPC Class Codes: Y10S 707/99944 Object-oriented database structure

Abstract

By sampling the initial result of a query to the database, thereby producing a sampled result which is substantially smaller than the initial result, the sampled result is easier to analyse statistically and is easier to transmit across a communications medium such as a network.

Claims

1. A method of retrieving and analysing data held in a database, comprising:

addressing a query to a database to generate a sequential stream of initial result records; and

sampling said sequential stream of initial result records to produce a sampled result which is substantially smaller than the initial result, the initial result being the result of a query to a database.

2. A method according to claim 1 wherein all records in the initial result stream are examined to determine whether they are sampled.

3. A method according to claim 2, wherein sampling is performed by successive iterations through a loop, each iteration comprising:

generating a random number within defined limits;

reading a record;

comparing the random number with a predetermined number to determine whether a record is sampled; and

selecting or rejecting the record accordingly.

4. A method according to claim 1, wherein random records in the initial result stream are sampled.

5. A method according to claim 4, wherein sampling is performed by selecting an initial record from the sequential stream of results and successive iterations through a loop, each iteration comprising:

generating a random skip count;

discarding a number of records equal to the skip count; and

selecting the next record.

6. A method according to claim 1, wherein sampling is performed during database access or as a database query engine builds a table of results.

7. A method according to claim 1, wherein sampling is performed after a table of results has been completed.

8. A method according to claim 1, wherein sampling is performed during the loading of a results table into an analysis tool.

9. A method according to claim 1, further comprising transmitting the sampled result across a communications medium.

10. A method according to claim 1, comprising transmitting the initial result across a communications medium before sampling.

11. A method according to claim 1, comprising treating each record as nrec identical records when the initial result stream contains averaged or aggregated or averaged and aggregated data, where nrec is equal to the number of records averaged or aggregated to produce the averaged or aggregated record.

12. A method according to claim 1, further comprising analysing the sampled result.

13. A method according to claim 12, wherein any one, or combination of, the operations grouping, aggregating and scaling are performed on the records of the analysed sampled result.

14. A method according to claim 1, wherein the number of records in the initial result is recorded.

15. A method according to claim 1, wherein each record in the initial result stream has a predetermined probability of being sampled.

16. A method according to claim 15, wherein the probability of selecting a record is weighted in accordance with a field in the record.

17. A method according to claim 16, wherein the probability of selecting a record is proportional to a field in the record.

18. A computer database system operating according to the method as defined in claim 1.

19. A method of process management comprising:

addressing a query to a database to generate a sequential stream of initial result records; and

sampling said sequential stream of initial result records to produce a sampled result which is substantially smaller than the initial result.