Editable and searchable markup pages automatically populated through user query monitoring

• US 9,244,952 B2
• Filed: 10/18/2013
• Issued: 01/26/2016
• Est. Priority Date: 03/17/2013
• Status: Active Grant

• Current Assignee: Alation, Inc.
• Original Assignee: Alation, Inc.
• Inventors: Ganti, Venkatesh, Kalb, Aaron, Niu, Feng, Sangani, Satyen
• Primary Examiner(s): Reyes, Mariela
• Assistant Examiner(s): Almani, Mohsen
• Application Number:
  US14/058,208
• Publication Number:
  US 20140279845A1
• Time in Patent Office: 830 Days
• Field of Search: 707/608
• US Class Current: 1/1
• CPC Class Codes: G06F 16/211 Schema design and management
• CPC Class Codes: G06F 16/219 Managing data history or versioning querying versioned data G06F16/2474; querying temporal data G06F16/2477
• CPC Class Codes: G06F 16/248 Presentation of query results
• CPC Class Codes: G06F 16/3325 Reformulation based on results of preceding query

Abstract

Disclosed are a method, a device and/or a system of editable and searchable markup pages automatically populated through query monitoring of users of a database. In one aspect, a method includes automatically generating an editable markup page and/or a page name based on an initial query of a database using a processor and a memory, associating the generated markup page with a user of the database, and appending information to the editable markup page based on a similar query of the database by another user. The method may include permitting other users of the database to access, modify, append, and/or delete entries from the editable mark-up page.

Claims

1. A method comprising:

automatically generating, through a processor communicatively coupled to a memory, an editable and searchable markup page based on an initial query of a database;

associating, through the processor, the generated editable and searchable markup page with a user of the database;

automatically extracting, through the processor, metadata of at least one data object of the database utilized by another query of the database from a corresponding at least one data source, the another query capable of being both the initial query and a query similar to the initial query, the another query being at least one of;

from another user of the database and a previous query from the user of the database, and the at least one data object comprising at least one of;

an attribute, a table, a schema, a predicate, an expression and an attribute value;

associating, through the processor, the automatically extracted metadata of the at least one data object with the initial query;

appending, through the processor, information to the editable and searchable markup page based on the association of the automatically extracted metadata of the at least one data object with the initial query;

enhancing, through the processor, the information appended to the editable and searchable markup page based on extracting an action related to the at least one data object by at least one knowledgeable user of the database;

generating, through the processor, a social data catalog that populates the editable and searchable markup page;

populating, through the processor, the social data catalog with metadata, a logical definition and description of attributes, information about usage, page views between users, a social data network, and a statistical data profile;

extracting, through the processor, information from external data sources and social media profiles to generate the social data catalog of information; and

enabling, through the processor, crowdsourcing of information from a ranked list of knowledgeable users to generate a ranked order of priority of information presented in profile pages of a curated answers system,

wherein the information about usage includes related tables and join predicates as well as relevant filters associated with each table of at least one of the database and a sample database,

wherein the social data network includes a list of users who are knowledgeable about a particular object related to the another query, and

wherein the information extracted is metadata comprising at least one of a schema name, a table in the schema, a name of the attribute, a data type of the attribute, a primary key associated with the attribute, a constraint of the attribute, a functional dependency between attributes, an index, a foreign key, a field name, a column name, a table name, and a query description.

2. The method of claim 1, further comprising at least one of:

permitting, through the processor, other users of the database to access the editable and searchable markup page;

permitting, through the processor, the other users of the database to modify the editable and searchable markup page;

permitting, through the processor, the other users of the database to append more information in the editable and searchable markup page; and

permitting, through the processor, the other users of the database to delete the information in the editable and searchable markup page.

3. The method of claim 2, further comprising:

tracking, through the processor, each edit made by the other users in a log file; and

presenting, through the processor, the log file on the editable and searchable markup page to enable visitors to the editable and searchable markup page view a change history thereof by the other users of the database.

4. The method of claim 1, further comprising:

creating, through the processor, a set of rules in which the user can flag an edit made by the another user of the database as being marked for deletion;

enabling, through the processor, other users of the database to vote on whether the edit made should be deleted; and

restoring, through the processor, the editable and searchable markup page to a state thereof prior to the edit being made based on a successful vote of the other users of the database.

5. The method of claim 1, further comprising at least one of:

automatically populating, through the processor, a profile markup page of the user with information describing the initial query;

determining, through the processor, semantic proximity of the another query to the initial query;

presenting, through the processor, the profile markup page of the user to the another user; and

automatically appending, through the processor, information about the another query semantically proximate to the initial query on the editable and searchable markup page.

6. The method of claim 5, further comprising at least one of:

enabling, through the processor, the another user to communicate with the user through a communication channel on the profile markup page;

publishing, through the processor, a question of the another user to the user on the profile markup page of the user and another profile markup page of the another user;

associating, through the processor, the question with the another user;

publishing, through the processor, a response of the user to the question on the profile markup page of the user, the another profile markup page of the another user, and on the editable and searchable markup page; and

associating, through the processor, the response with the user.

7. The method of claim 6, further comprising:

automatically populating, through the processor, the another profile markup page of the another user with information describing the another query,

wherein the automatic population of the profile markup page and the another profile markup page are both performed through automatic observation and monitoring of activity of the user and the another user in interacting with the database.

8. The method of claim 1, further comprising permitting, through the processor, users of the database to associate at least one of an electronic mailing address, a phone number, a biography, a personal photograph, a social media handle, and an educational background with a profile thereof associated with the editable and searchable markup page.

9. The method of claim 1, further comprising at least one of:

associating, through the processor, relationships between different data objects comprising at least one of popular tables, popular attributes, co-queried attributes in a table, related tables, joinable columns, joinable predicates, and a relevant filter for each table using the social data catalog generated;

associating, through the processor, relationships between data objects and users comprising a list of knowledgeable people to be contacted about particular objects using the social data catalog;

parsing, through the processor, queries into constituent fragments; and

aggregating, normalizing, and storing, through the processor, results of the parsed queries.

10. The method of claim 9, further comprising at least one of:

mining, through the processor, information using the results of the parsed queries; and

populating, through the processor, the social data catalog associated with the editable and searchable markup page with information from at least one of crawled query logs, analyzed application logs, and a query editing tool based on the mining,

wherein crawling query logs ingest queries from files where users of the database store queries,

wherein application logs are analyzed for queries, and

wherein query editing tools are added to the social data catalog to develop and append developed queries to the editable and searchable markup page.

11. The method of claim 10, further comprising:

implementing, through a social data catalog module associated with the processor, at least one of a materialized computation and an on-demand computation as an alternative to an Application Programming Interface (API) function,

wherein the materialized computation involves periodically mining, through the processor, query logs to update a query reference table with new queries that have not been previously processed as well as materializing and indexing derived information of different data objects, and

wherein on-demand computation involves indexing, through the processor, the query reference table using various objects that each row references.

12. The method of claim 9, further comprising:

associating, through the processor, a data catalog studio with the editable and searchable markup page; and

generating, through the processor, in the data catalog studio a data catalog repository, a data source registrar, a background extraction component, a frontend component, and a user interaction logger;

wherein the data catalog repository is configured to;

to store the social data catalog,

to store a data object and information about the data object, and to return information about the data object,

to update stored information, and to return a ranked list of relevant data objects, and

to search and to return a list of data objects of a given type.

13. The method of claim 12, further comprising at least one of:

registering, through the processor, the at least one data source and extracting, through the processor, declared metadata from a schema using the data source registrar;

enabling, through the processor, at least one of the user and the another user to register the at least one data source, upload documentation on the at least one data source, upload a query log, search relevant objects, and browse the schema in the at least one data source using the front end component;

automatically profiling, locating, extracting, and associating, through the processor, a documentation of the at least one of the user and the another user in the at least one data source using the background extraction component;

automatically profiling, locating, extracting, and associating, through the processor, a query log of the at least one of the user and the another user in the at least one data source using the background extraction component; and

monitoring and logging, through the processor, interactions between the front end component and various users accessing the front end component to add, delete, reorder, modify, and sort information presented in profile pages of users of a curated answers system.

14. A non-transitory medium, readable through at least one processor communicatively coupled to at least one memory and comprising instructions embodied therein that are executable through the at least one processor, comprising:

instructions to automatically generate an editable and searchable markup page based on an initial query of a database;

instructions to associate the generated editable and searchable markup page with a user of the database;

instructions to automatically extract metadata of at least one data object of the database utilized by another query of the database from a corresponding at least one data source, the another query capable of being both the initial query and a query similar to the initial query, the another query being at least one of;

from another user of the database and a previous query from the user of the database, and the at least one data object comprising at least one of;

an attribute, a table, a schema, a predicate, an expression and an attribute value;

instructions to associate the automatically extracted metadata of the at least one data object with the initial query;

instructions to append information to the editable and searchable markup page based on the association of the automatically extracted metadata of the at least one data object with the initial query;

instructions to enhance the information appended to the editable and searchable markup page based on extracting an action related to the at least one data object by at least one knowledgeable user of the database,

instructions to generate a social data catalog that populates the editable and searchable markup page;

instructions to populate the social data catalog with metadata, a logical definition and description of attributes, information about usage, page views between users, a social data network, and a statistical data profile;

instructions to extract information from external data sources and social media profiles to generate the social data catalog of information; and

instructions to enable crowdsourcing of information from a ranked list of knowledgeable users to generate a ranked order of priority of information presented in profile pages of a curated answers system,

wherein the information about usage includes related tables and join predicates as well as relevant filters associated with each table of at least one of the database and a sample database,

wherein the social data network includes a list of users who are knowledgeable about a particular object related to the another query, and

wherein the information extracted is metadata comprising at least one of a schema name, a table in the schema, a name of the attribute, a data type of the attribute, a primary key associated with the attribute, a constraint of the attribute, a functional dependency between attributes, an index, a foreign key, a field name, a column name, a table name, and a query description.

15. The non-transitory medium of claim 14, further comprising at least one of:

instructions to permit other users of the database to access the editable and searchable markup page;

instructions to permit the other users of the database to modify the editable and searchable markup page;

instructions to permit the other users of the database to append more information in the editable and searchable markup page; and

instructions to permit the other users of the database to delete the information in the editable and searchable markup page.

16. The non-transitory medium of claim 15, further comprising:

instructions to track each edit made by the other users in a log file; and

instructions to present the log file on the editable and searchable markup page to enable visitors to the editable and searchable markup page view a change history thereof by the other users of the database.

17. The non-transitory medium of claim 14, further comprising:

instructions to create a set of rules in which the user can flag an edit made by the another user of the database as being marked for deletion;

instructions to enable other users of the database vote on whether the edit made should be deleted; and

instructions to restore the editable and searchable markup page to a state thereof prior to the edit being made based on a successful vote of the other users of the database.

18. A system comprising:

at least one memory; and

at least one processor communicatively coupled to the at least one memory, the at least one processor being configured to execute instructions to;

automatically generate an editable and searchable markup page based on an initial query of a database,

associate the generated editable and searchable markup page with a user of the database,

automatically extract metadata of at least one data object of the database utilized by another query of the database from a corresponding at least one data source, the another query capable of being both the initial query and a query similar to the initial query, the another query being at least one of;

from another user of the database and a previous query from the user of the database, and the at least one data object comprising at least one of;

an attribute, a table, a schema, a predicate, an expression and an attribute value,

associate the automatically extracted metadata of the at least one data object with the initial query,

append information to the editable and searchable markup page based on the association of the automatically extracted metadata of the at least one data object with the initial query,

enhance the information appended to the editable and searchable markup page based on extracting an action related to the at least one data object by at least one knowledgeable user of the database,

generate a social data catalog that populates the editable and searchable markup page,

populate the social data catalog with metadata, a logical definition and description of attributes, information about usage, page views between users, a social data network, and a statistical data profile,

extract information from external data sources and social media profiles to generate the social data catalog of information, and

enable crowdsourcing of information from a ranked list of knowledgeable users to generate a ranked order of priority of information presented in profile pages of a curated answers system,

wherein the information about usage includes related tables and join predicates as well as relevant filters associated with each table of at least one of the database and a sample database,

wherein the social data network includes a list of users who are knowledgeable about a particular object related to the another query, and

wherein the information extracted is metadata comprising at least one of a schema name, a table in the schema, a name of the attribute, a data type of the attribute, a primary key associated with the attribute, a constraint of the attribute, a functional dependency between attributes, an index, a foreign key, a field name, a column name, a table name, and a query description.

19. The system of claim 18, wherein the at least one processor is further configured to execute instructions to at least one of:

permit other users of the database to access the editable and searchable markup page,

permit the other users of the database to modify the editable and searchable markup page,

permit the other users of the database to append more information in the editable and searchable markup page, and

permit the other users of the database to delete the information in the editable and searchable markup page.

20. The system of claim 19, wherein the at least one processor is further configured to execute instructions to:

track each edit made by the other users in a log file, and

present the log file on the editable and searchable markup page to enable visitors to the editable and searchable markup page view a change history thereof by the other users of the database.