System and method for web knowledge extraction

• US 8,918,357 B2
• Filed: 07/26/2011
• Issued: 12/23/2014
• Est. Priority Date: 07/26/2011
• Status: Expired due to Fees

• Current Assignee: R2 Solutions LLC (Acacia Research Corporation)
• Original Assignee: Yahoo! Inc. (Apollo Global Management, Inc.)
• Inventors: Minocha, Hemant, Ahmed, Fakrudeen Ali, Ragunathan, Venkatavaradan, Verma, Vikram, Rajpurohit, Gopal Mishrimalji, Suprasadachandranpillai, Syama Prasad, Rajadurai A, Tom Praison, Sortur, Muralidhar Hanumantachar, Alle, Sachidanand, Baranwal, Ramashish
• Primary Examiner(s): Dwivedi, Mahesh
• Assistant Examiner(s): MACKES, KRIS E
• Application Number:
  US13/191,231
• Publication Number:
  US 20130031131A1
• Time in Patent Office: 1,246 Days
• Field of Search: 707/602
• US Class Current: 707/602
• CPC Class Codes: G06F 16/00 Information retrieval; Database structures therefor; File system structures therefor
• CPC Class Codes: G06F 16/95 Retrieval from the web
• CPC Class Codes: G06Q 10/06 Resources, workflows, human or project management; Enterprise or organisation planning; Enterprise or organisation modelling
• CPC Class Codes: G06Q 30/02 Marketing; Price estimation or determination; Fundraising

Abstract

Embodiments of the disclosed invention include an apparatus, method, and computer program product for creating and executing a client workflow for web data extraction. For example, the disclosed embodiments provide a system for web data extraction. The system includes a data storage component configured for storing a plurality of preconfigured reusable software components that provide services for creating a client workflow for web data extraction. The system also includes a communication interface operable to receive workflow definitions from a client for creating the client workflow for web data extraction utilizing at least one of the plurality of preconfigured reusable software components. The system has a processor for executing instructions to run the client workflow for web data extraction.

Claims

1. A system for web data extraction, comprising:

a non-transitory data storage component configured for storing a plurality of preconfigured reusable software components that provide services for creating a client workflow for web data extraction;

a communication interface operable to receive information related to workflow from a client for creating the client workflow for web data extraction utilizing at least one of the plurality of preconfigured reusable software components; and

a processor for executing instructions to run the client workflow for web data extraction, wherein the plurality of preconfigured reusable software components includes a web data surfacing component configured for extracting web data from a plurality of web data extraction services, the web data surfacing component comprising instructions for formatting the request in at least one format to generate the at least one formatted request, communicating the at least one formatted request to the at least one of the plurality of web data extraction services, and receiving a response from the at least one of the plurality of web data extraction services.

2. The system of claim 1, wherein the client workflow includes at least one custom workflow component provided by the client for providing services for custom logic for web data extraction.

3. The system of claim 1, wherein the plurality of preconfigured reusable software components includes a grid processing component.

4. The system of claim 2, wherein the plurality of preconfigured reusable software components includes a designer interface in communication with a designer component, wherein a client selects the plurality of custom components and provides the information related to workflow using the designer component for configuring the client workflow for web data extraction.

5. The system of claim 1, wherein the plurality of preconfigured reusable software components includes a dashboard interface for providing workflow monitoring services to a dashboard component.

6. The system of claim 1, wherein the web data surfacing component comprises:

instructions for receiving a request to extract web data; and

instructions for combining the response from the at least one of the web data extraction services into a combined formatted response for satisfying the request.

7. The system of claim 6, wherein the instructions for formatting the request in at least one format associated with at least one of the plurality of web data extraction services comprises:

instructions for identifying requirements of the request; and

instructions for identifying the at least one of the plurality of data extraction services capable of satisfying the requirements of the request based on parameters of each of the plurality of data extraction services.

8. The system of claim 6, wherein the processor further executes instructions to transfer the response received from the at least one of the plurality of data extraction services in batches for enabling workload processing to occur on received batches of web data while awaiting additional batches of web data from the at least one of the plurality of data extraction services.

9. The system of claim 6, wherein the processor further executes instructions to transfer the response received from the at least one of the plurality of data extraction services to a distributed grid for workload processing.

10. The system of claim 6, wherein the processor further executes instructions to validate the output of the client workflow for web data extraction, wherein the output is in one of a plurality of structured formats.

11. A method, implemented on a machine having at least one processor, storage, and a communication platform connected to a network for web data extraction, the method comprising:

receiving information related to workflow from a client device for at least one of a plurality of preconfigured reusable software components that provide services for creating a client workflow for web data extraction;

generating the client workflow based on the information related to workflow; and

extracting web data in accordance with the client workflow which comprises;

formatting a web data extraction request in at least one format to generate the at least one formatted request,

communicating the at least one formatted request to the at least one of the plurality of web data extraction services, and

receiving a response from the at least one of the plurality of web data extraction services that received the at least one formatted request.

12. The method of claim 11 further comprising:

receiving at least one custom workflow component from the client device for providing services for custom logic for web data extraction; and

wherein generating the client workflow includes generating the client workflow based on the information related to workflow and the at least one custom workflow component.

13. The method of claim 11 further comprising:

transferring the response received from the at least one of the plurality of data extraction services to a distributed grid for workload processing.

14. The method of claim 11 further comprising:

validating the output of the client workflow against at least one rule for verifying a structure of the output.

15. A machine-readable tangible and non-transitory medium having information for web data extraction, wherein the information, when read by the machine, causes the machine to perform the following:

receive information related to workflow from a client device for at least one of a plurality of preconfigured reusable software components that provide services for creating a client workflow for web data extraction;

generate the client workflow based on the information related to workflow;

extract web data in accordance with the client workflow;

format a web data extraction request in at least one format to generate the at least one formatted request;

communicate the at least one formatted request to the at least one of the plurality of web data extraction services; and

receive a response from the at least one of the plurality of web data extraction services that received the at least one formatted request.

16. The medium of claim 15, wherein the information, when read by the machine, further causes the machine to:

receive at least one custom workflow component from the client device for providing services for custom logic for web data extraction; and

generate the client workflow based on the information related to workflow and the at least one custom workflow component.

17. The medium of claim 15, wherein the information, when read by the machine, further causes the machine to:

transfer the response received from the at least one of the plurality of data extraction services to a distributed grid for workload processing.