SYSTEM AND METHOD FOR WEB KNOWLEDGE EXTRACTION

• US 20130031131A1
• Filed: 07/26/2011
• Published: 01/31/2013
• Est. Priority Date: 07/26/2011
• Status: Active Grant

• Current Assignee: R2 Solutions LLC (Acacia Research Corporation)
• Original Assignee: Yahoo! Inc. (Apollo Global Management, Inc.)
• Inventors: Minocha, Hemant, Ahmed, Fakrudeen Ali, Ragunathan, Venkatavaradan, Verma, Vikram, Rajpurohit, Gopal Mishrimalji, Suprasadachandranpillai, Syama Prasad, Rajadurai A, Tom Praison, Sortur, Muralidhar Hanumantachar, Alle, Sachidanand, Baranwal, Ramashish
• Granted Patent: US 8,918,357 B2
• Time in Patent Office: Days
• US Class Current: 707/770
• CPC Class Codes: G06F 16/00 Information retrieval; Database structures therefor; File system structures therefor
• CPC Class Codes: G06F 16/95 Retrieval from the web
• CPC Class Codes: G06Q 10/06 Resources, workflows, human or project management; Enterprise or organisation planning; Enterprise or organisation modelling
• CPC Class Codes: G06Q 30/02 Marketing; Price estimation or determination; Fundraising

Abstract

Embodiments of the disclosed invention include an apparatus, method, and computer program product for creating and executing a client workflow for web data extraction. For example, the disclosed embodiments provide a system for web data extraction. The system includes a data storage component configured for storing a plurality of preconfigured reusable software components that provide services for creating a client workflow for web data extraction. The system also includes a communication interface operable to receive workflow definitions from a client for creating the client workflow for web data extraction utilizing at, least one of the plurality of preconfigured reusable software components. The system has a processor for executing instructions to run the client workflow for web data extraction.

Claims

1. A system for web data extraction, comprising:

a data storage component configured for storing a plurality of preconfigured reusable software components that provide services for creating a client workflow for web data extraction;

a communication interface operable to receive workflow definitions from a client for creating the client workflow for web data extraction utilizing at least one of the plurality of preconfigured reusable software components; and

a processor for executing instructions to run the client workflow for web data extraction.

2. The system of claim 1, wherein the client workflow includes at least one custom workflow component provided by the client for providing services for custom logic for web data extraction.

3. The system of claim 1, wherein the plurality of preconfigured reusable software components includes a grid processing component.

4. The system of claim 2, wherein the plurality of preconfigured reusable software components includes a designer interface in communication with a designer component, wherein a client selects the plurality of custom components and provides the workflow definitions using the designer component for configuring the client workflow for web data extraction.

5. The system of claim 1, wherein the plurality of preconfigured reusable software components includes a dashboard interface for providing workflow monitoring services to a dashboard component.

6. The system of claim 1, wherein the plurality of preconfigured reusable software components includes a web data surfacing component for extracting web data from a plurality of web data extraction services.

7. The system of claim 6, wherein the web data surfacing component comprises:

instructions for receiving a request to extract web data;

instructions for formatting the request in at least one format associated with at least one of the plurality of web data extraction services;

instructions for communicating the at least one formatted request to the at least one of the plurality of web data extraction services;

instructions for receiving a response from the at least one of the plurality of web data extraction services that received the at least one formatted request; and

instructions for combining the response from the at least one of the web data extraction services into a combined formatted response for satisfying the request.

8. The system of claim 7, wherein the instructions for formatting the request in at least one format associated with at least one of the plurality of web data extraction services comprises:

instructions for identifying requirements of the request; and

instructions for identifying the at least one of the plurality of data extraction services capable of satisfying the requirements of the request based on parameters of each of the plurality of data extraction services.

9. The system of claim 7, wherein the processor further executes instructions to transfer the response received from the at least one of the plurality of data extraction services in batches for enabling workload processing to occur on received batches of web data while awaiting additional batches of web data from the at least one of the plurality of data extraction services.

10. The system of claim 7, wherein the processor further executes instructions to transfer the response received from the at least one of the plurality of data extraction services to a distributed grid for workload processing.

11. The system of claim 7, wherein the processor further executes instructions to validate the output of the client workflow for web data extraction, wherein the output is in one of a plurality of structured formats.

15. The method of claim 11 further comprising:

transferring the response received from the at least one of the plurality of data extraction services to a distributed grid for workload processing.

16. The method of claim 11 further comprising:

validating the output of the client workflow against at least one rule for verifying a structure of the output.

12. A method, implemented on a machine having at least one processor, storage, and a communication platform connected to a network for web data extraction, the method comprising the steps of

receiving workflow definitions from a client device for at least one of a plurality of preconfigured reusable software components that provide services for creating a client workflow for web data extraction;

generating the client workflow using the workflow definitions; and

executing the client workflow for web data extraction.

13. The method of claim 12 further comprising:

receiving at least one custom workflow component from the client device for providing services for custom logic for web data extraction; and

wherein generating the client workflow includes generating the client workflow using the workflow definitions and the at least one custom workflow component.

14. The method of claim 12, wherein executing the client workflow for web data extraction comprises:

formatting a web data extraction request in at least one format associated with at least one of a plurality of web data extraction services;

communicating the at least one formatted request to the at least one of the plurality of web data extraction services; and

receiving a response from the at least one of the plurality of web data extraction services that received the at least one formatted request.

17. A machine-readable tangible and non-transitory medium having information for web data extraction, wherein the information, when read by the machine, causes the machine to perform the following:

receive workflow definitions from a client device for at least one of a plurality of preconfigured reusable software components that provide services for creating a client workflow for web data extraction;

generate the client workflow using the workflow definitions; and

execute the client workflow for web data extraction.

18. The method of claim 17, wherein the information, when read by the machine, further causes the machine to:

receive at least one custom workflow component from the client device for providing services for custom logic for web data extraction; and

generate the client workflow using the workflow definitions and the at least one custom workflow component.

19. The medium of claim 17, wherein the information, when read by the machine, further causes the machine to:

format a web data extraction request in at least one format associated with at least one of a plurality of web data extraction services;

communicate the at least one formatted request to the at least one of the plurality of web data extraction services; and

receive a response from the at least one of the plurality of web data extraction services that received the at least one formatted request.

20. The medium of claim 17, wherein the information, when read by the machine, further causes the machine to:

transfer the response received from the at least one of the plurality of data extraction services to a distributed grid for workload processing