System and methods for processing large datasets
First Claim
1. A method of organizing and generating report data renderable on a graphical user output device indicative of a plurality of manageable entities in a managed information network comprising:
- determining an output criteria applicable to a set of data, the set of data having entries including fields, to compute an output display set adapted for inclusion in a report, the determined output criteria including;
a selection criteria indicative of output records for inclusion in a set of candidate entries selected from the entries; and
an arrangement criteria indicative of display formatting applicable to the candidate entries;
parsing, in a first pass, a plurality of entries in the set of data;
applying, during the first pass, the selection criteria to identify a subset of entries from among the entries in the set of data for inclusion as output records in the report, the selection criteria applying a filtering criteria indicating the fields in the entries used for an output display set to compute the subset of candidate entries;
the arrangement criteria indicating a logical arrangement order applicable to the subset of candidate entries identified from the first pass;
retaining in a memory, in the first pass, only the fields used in the selection and arrangement criteria;
parsing, during the second pass, the computed subset of candidate entries, the second pass performed on the same set of data unmodified from the first pass;
omitting from a memory, in the second pass, entries in the parsed plurality of entries outside the subset of candidate entries;
processing, during the second pass, the computed subset of candidate entries according to the arrangement criteria to generate the output display set of entries on the graphical user output device;
the determined output criteria operable on key fields of the entries, the key fields having scalar values operable for comparison, and the set of data being arranged in a predetermined data set format, the data set format operable to designate entries corresponding to rows and fields corresponding to columns.
9 Assignments
0 Petitions
Accused Products
Abstract
In a storage area network (SAN) management application, large data sets tend to produce unwieldy results upon display attempts. A mechanism for gathering and organizing a large data set into an ordered set of output display entries which does not require main memory storage for the entire large data set, generates a displayable portion without excessive memory consumption. Configurations employ a multiple pass gathering and organization operation. An output criteria specifies a key field and identifies a range of entries by specifying a particular screen for display from among the full range of screens. A parser retrieves and identifies the entries, during a first pass, using only the key field and an index for each of the entries in the large data set, to avoid loading all fields of each entry in entirety. A second pass fetches entries only for the identified entries corresponding to the output criteria.
29 Citations
26 Claims
-
1. A method of organizing and generating report data renderable on a graphical user output device indicative of a plurality of manageable entities in a managed information network comprising:
-
determining an output criteria applicable to a set of data, the set of data having entries including fields, to compute an output display set adapted for inclusion in a report, the determined output criteria including; a selection criteria indicative of output records for inclusion in a set of candidate entries selected from the entries; and an arrangement criteria indicative of display formatting applicable to the candidate entries; parsing, in a first pass, a plurality of entries in the set of data; applying, during the first pass, the selection criteria to identify a subset of entries from among the entries in the set of data for inclusion as output records in the report, the selection criteria applying a filtering criteria indicating the fields in the entries used for an output display set to compute the subset of candidate entries; the arrangement criteria indicating a logical arrangement order applicable to the subset of candidate entries identified from the first pass; retaining in a memory, in the first pass, only the fields used in the selection and arrangement criteria; parsing, during the second pass, the computed subset of candidate entries, the second pass performed on the same set of data unmodified from the first pass; omitting from a memory, in the second pass, entries in the parsed plurality of entries outside the subset of candidate entries; processing, during the second pass, the computed subset of candidate entries according to the arrangement criteria to generate the output display set of entries on the graphical user output device; the determined output criteria operable on key fields of the entries, the key fields having scalar values operable for comparison, and the set of data being arranged in a predetermined data set format, the data set format operable to designate entries corresponding to rows and fields corresponding to columns. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 9, 10, 11)
-
-
12. A data management device for organizing and generating report data indicative of a plurality of manageable entities renderable on a graphical user output device in a storage area network (SAN) comprising:
-
an infrastructure monitor operable for receiving an output criteria applicable to a set of data, the set of data having entries including fields, to compute an ordered output display set adapted for inclusion in a report, the received output criteria including; a selection criteria to identify a subset of entries from among the entries in the set of data for inclusion as output records in the report, the selection criteria applying a filtering criteria indicating the fields in the entries used for an output display set to compute a subset of candidate entries; and an arrangement criteria indicative of a logical arrangement order applicable to the subset of candidate entries identified from the first pass, the set of data arranged in a predetermined data set format, the data set format operable to designate entries corresponding to rows and fields corresponding to columns; a parser in the infrastructure monitor for parsing, in a first pass, a plurality of entries in the set of data, the parser operable to omit from a memory, in the second pass, entries in the parsed plurality of entries outside the subset of candidate entries; a first handler in the parser for applying, during the first pass, the selection criteria to identify a subset of entries from among the entries in the set of data for inclusion as output records in the report, the selection criteria applying a filtering criteria indicating the fields in the entries used for an output display set to compute the subset of candidate entries; a memory for retaining, in the first pass, only the selection and arrangement criteria fields; a second handler in the parser for applying, during a second pass, the arrangement criteria to the subset of candidate entries, the arrangement criteria indicating a logical arrangement order applicable to the subset of candidate entries identified from the first pass, the parser operable to process, during the second pass, the computed subset of candidate entries according to the determined output criteria to generate the ordered output display set of entries on the graphical user output device; the determined output criteria operable on key fields of the entries, the key fields having scalar values operable for comparison, and the set of data being arranged in a predetermined data set format, the data set format operable to designate entries corresponding to rows and fields corresponding to columns. - View Dependent Claims (13, 14, 15, 16, 17, 18, 19, 20)
-
-
21. A method for efficient memory usage for organizing and generating report data indicative of a plurality of manageable entities in a storage area network (SAN) comprising:
-
determining, via a user input request, an output criteria applicable to a set of data, the set of data having entries including fields, to compute an ordered output display set adapted for inclusion in a report; parsing, by a build set handler, in a first pass, a plurality of entries in the set of data; filtering according to a filtering criteria, the filtering operable to designate a subset of entries for inclusion in the first pass and the filtering criteria operable on at least one of the fields for comparison and selective inclusion in the designated subset; retrieving, during the parsing, a portion of each of the plurality of entries, the filtering criteria defining the portion as fields in the entries deterministic of the entries in the output display set, wherein the parsed entries in the first pass are inoperably large for concurrent memory residence; applying, during the first pass, a selection criteria in the determined output criteria, the selection criteria for identifying a subset of entries from among the entries in the set of data for inclusion as output records in the report, the selection criteria applying a filtering criteria indicating the fields in the entries used for an output display set to compute the subset of candidate entries, the first pass further comprising filtering entries for omission from the candidate set and comparing key fields in entries for inclusion in the candidate set; retaining in a memory, in the first pass, only the selection and arrangement criteria fields; parsing, by a match set handler, in a second pass, the computed subset of candidate entries to identity entries in the data set corresponding to the portions of entries in the candidate entries, the second pass applying an arrangement criteria indicating a logical arrangement order applicable to the subset of candidate entries identified from the first pass and performed on the same set of data unmodified from the first pass; retrieving, by the match set handler, the entire entry corresponding to each of the portion of the entry in the candidate entries; processing, by a sort handler, during the second pass, the computed subset of candidate entries according to the determined output criteria to generate complete entries of the entries in the candidate set, wherein the parsed entries of the second pass require less memory than the first pass such that the parsed entries of the second pass are stored simultaneously in memory, the second pass further comprising matching entries for inclusion in the candidate set, sorting entries in the candidate set, and formatting entries in the candidate set for display on the output device; formatting, by a format handler, the processed computed subset of entries to generate the output display set of entries on a graphical user output device, the output display set of entries corresponding to the computed subset of entries and ordered according to the ordering logic; omitting from the memory, during the second pass, entries in the data set outside the output display set of entries; the determined output criteria operable on key fields of the entries, the key fields having scalar values operable for comparison, and the set of data being arranged in a predetermined data set format, the data set format operable to designate entries corresponding to rows and fields corresponding to columns.
-
-
22. A computer program product having a computer readable medium operable to store computer program logic embodied in an encoded set of processor based instructions encoded thereon for performing, via a processor responsive to the instructions, organizing and generating report data renderable on a graphical user output device indicative of a plurality of manageable entities in a managed information network comprising:
-
computer program code for determining an output criteria applicable to a set of data to compute an ordered output display set adapted for inclusion in a report, the set of data having entries including fields, the determined output criteria including; a selection criteria for identifying, in a first pass, a subset of entries from among the entries in the set of data for inclusion as output records in the report, the selection criteria applying a filtering criteria indicating the fields in the entries used for an output display set to compute the subset of candidate entries; and an arrangement criteria indicative of display formatting applicable to the candidate entries, the arrangement criteria operable in a second pass for indicating a logical arrangement order applicable to the subset of candidate entries identified from the first pass; computer program code for parsing, in the first pass, a plurality of entries in the set of data; computer program code for applying, during the first pass, the determined output criteria to a portion of each parsed entry to compute a subset of candidate entries, the output criteria defining the portion as fields in the entries deterministic of the entries in the output records; retaining in a memory, in the first pass, only the output criteria fields; computer program code for parsing, in the second pass, the computed subset of candidate entries, the second pass performed on the same set of data unmodified from the first pass; computer program code for omitting from a memory, in the second pass, entries in the parsed plurality of entries outside the candidate set; and computer program code for processing, during the second pass, the computed subset of candidate entries according to the determined output criteria to generate the ordered output display set of entries on the graphical user output device; the determined output criteria operable on key fields of the entries, the key fields having scalar values operable for comparison, and the set of data being arranged in a predetermined data set format, the data set format operable to designate entries corresponding to rows and fields corresponding to columns.
-
-
23. A data management device for organizing and generating report data indicative of a plurality of manageable entities renderable on a graphical user output device in a storage area network (SAN) comprising:
-
means for determining an output criteria applicable to a set of data, the set of data having entries including fields, to compute an ordered output display set adapted for inclusion in a report, the determined output criteria including; a selection criteria to identify, in a first pass, a subset of entries from among the entries in the set of data for inclusion as output records in the report, the selection criteria applying a filtering criteria indicating the fields in the entries used for an output display set to compute the subset of candidate entries; and an arrangement criteria indicative of display formatting applicable to the candidate entries, the arrangement criteria operable in a second pass for indicating a logical arrangement order applicable to the subset of candidate entries identified from the first pass; means for parsing, in the first pass, a plurality of entries in the set of data; means for applying, during the first pass, the determined output criteria to a portion of each parsed entry to compute a subset of candidate entries, the output criteria defining the portion as fields in the entries deterministic of the entries in the output records; retaining in a memory, in the first pass, only the output criteria fields; means for parsing, in the second pass, the computed subset of candidate entries, the second pass performed on the same set of data unmodified from the first pass; means for omitting from a memory, in the second pass, entries in the parsed plurality of entries outside the candidate set; and means for processing, during the second pass, the computed subset of candidate entries according to the determined output criteria to generate the ordered output display set of entries in a tangible medium of expression on the graphical user output device; the determined output criteria operable on key fields of the entries, the key fields having scalar values operable for comparison, and the set of data being arranged in a predetermined data set format, the data set format operable to designate entries corresponding to rows and fields corresponding to columns.
-
-
24. A method for organizing and generating report data indicative of a plurality of manageable entities renderable on a graphical user output device in a managed information network comprising:
-
determining an output criteria applicable to a set of data, the set of data having entries including fields, to compute an ordered output display set adapted for inclusion in a report, the determined output criteria including; a selection criteria operable during a first pass to identify a subset of entries from among the entries in the set of data for inclusion as output records in the report, the selection criteria applying a filtering criteria indicating the fields in the entries used for an output display set to compute a subset of candidate entries; and an arrangement criteria operable during a second pass for indicating a logical arrangement order applicable to the subset of candidate entries identified from the first pass for determining display formatting applicable to the candidate entries; parsing, in a first pass, a plurality of entries in the set of data; retaining in a memory, in the first pass, only the fields referenced by the selection and arrangement criteria; applying, during the first pass, the determined output criteria to a portion of each parsed entry to compute a subset of candidate entries, the output criteria defining the portion as fields in the entries deterministic of the entries in the output records; parsing, in the second pass, the computed subset of candidate entries, the parsed entries of the second pass requiring less memory than the first pass such that the parsed entries of the second pass are stored simultaneously in memory, the second pass performed on the same set of data unmodified from the first pass; omitting from a memory, in the second pass, entries in the parsed plurality of entries outside the candidate set; and processing, during the second pass, the computed subset of candidate entries according to the determined output criteria to generate the ordered output display set of entries on a graphical user output device; the determined output criteria operable on key fields of the entries, the key fields having scalar values operable for comparison, and the set of data being arranged in a predetermined data set format, the data set format operable to designate entries corresponding to rows and fields corresponding to columns. - View Dependent Claims (25, 26)
-
Specification