Method and apparatus for building sales tools by mining data from websites

• US 8,359,307 B2
• Filed: 04/18/2011
• Issued: 01/22/2013
• Est. Priority Date: 12/23/2005
• Status: Expired due to Fees

• Current Assignee: AT&T Intellectual Property II LP (AT&T, Inc.)
• Original Assignee: AT&T Intellectual Property II LP (AT&T, Inc.)
• Inventors: Bangalore, Srinivas, Feng, Junlan, Gilbert, Mazin, Wilpon, Jay Gordon
• Primary Examiner(s): Wong, Leslie
• Application Number:
  US13/088,935
• Publication Number:
  US 20110258531A1
• Time in Patent Office: 645 Days
• Field of Search: 707/726, 707/708, 707/710, 707/999.107, 707/999.01, 709/203, 709/219
• US Class Current: 707/708
• CPC Class Codes: G06F 40/134 Hyperlinking
• CPC Class Codes: G06Q 30/02 Marketing; Price estimation or determination; Fundraising
• CPC Class Codes: G06Q 30/06 Buying, selling or leasing transactions

Abstract

A website mining tool is disclosed that extracts information from, for example, a company'"'"'s website and presents the extracted information in a graphical user interface (GUI). In one embodiment, web pages from a website are stored in, for example, computer memory and a structure of the web pages is identified. A plurality of blocks of information is then extracted as a function of this structure and a category is assigned to each block of information. The elements in the blocks of information are then displayed, for example to a salesperson, as a function of these categories. In another embodiment, Document Object Modeling parsing is used to identify the structure of the web pages. In yet another embodiment, a support vector machine is used to categorize each block of information.

Claims

1. A method for characterizing a plurality of extensible markup language documents, each of the plurality of extensible markup language documents comprising a link to another extensible markup language document of the plurality of extensible markup language documents, the method comprising:

traversing the plurality of extensible markup language documents by following each link of the plurality of extensible markup language documents;

parsing the plurality of extensible markup language documents to determine a structural hierarchy of the plurality of extensible markup language documents;

associating a task complexity with the plurality of extensible markup language documents based on the structural hierarchy, the associating comprising;

extracting a plurality of blocks of information from the plurality of extensible markup language documents; and

assigning a block of information in the plurality of blocks of information to a category in a plurality of categories, the plurality of categories comprising a task complexity category, the block of information comprising a value indicative of a number of extensible markup language documents associated with the plurality of extensible markup language documents and a value indicative of a number of links associated with the plurality of extensible markup language documents; and

characterizing the plurality of extensible markup language documents based on the task complexity.

2. The method of claim 1, further comprising:

displaying the structural hierarchy of the plurality of extensible markup language documents.

3. The method of claim 1 wherein parsing the plurality of extensible markup language documents to determine a structural hierarchy of the plurality of extensible markup language documents comprises:

parsing the plurality of extensible markup language documents using document object model parsing.

4. The method of claim 1 wherein parsing the plurality of extensible markup language documents to determine a structural hierarchy of the plurality of extensible markup language documents comprises:

downloading the extensible markup language documents to a local disk.

5. The method of claim 1 wherein the plurality of blocks of information comprise information identifying one of a company'"'"'s products.

6. The method of claim 5 wherein the plurality of blocks of information further comprise contact information.

7. A system for characterizing a plurality of extensible markup language documents, each of the plurality of extensible markup language documents comprising a link to another extensible markup language document of the plurality of extensible markup language documents, the system comprising:

means for traversing the plurality of extensible markup language documents by following each link of the plurality of extensible markup language documents;

means for parsing the plurality of extensible markup language documents to determine a structural hierarchy of the plurality of extensible markup language documents;

means for associating a task complexity with the plurality of extensible markup language documents based on the structural hierarchy, the means for associating comprising;

means for extracting a plurality of blocks of information from the plurality of extensible markup language documents; and

means for assigning a block of information in the plurality of blocks of information to a category in a plurality of categories, the plurality of categories comprising a task complexity category, the block of information comprising a value indicative of a number of extensible markup language documents associated with the plurality of extensible markup language documents and a value indicative of a number of links associated with the plurality of extensible markup language documents; and

means for characterizing the plurality of extensible markup language documents based on the task complexity.

8. The system of claim 7, further comprising:

means for displaying the structural hierarchy of the plurality of extensible markup language documents.

9. The system of claim 7 wherein means for parsing the plurality of extensible markup language documents to determine a structural hierarchy of the plurality of extensible markup language documents comprises:

means for parsing the extensible markup language documents using document object model parsing.

10. The system of claim 7 wherein means for parsing the plurality of extensible markup language documents to determine a structural hierarchy of the plurality of extensible markup language documents comprises:

means for downloading the extensible markup language documents to a local disk.

11. A non-transitory computer readable medium storing computer program instructions for characterizing a plurality of extensible markup language documents, each of the plurality of extensible markup language documents comprising a link to another extensible markup language document of the plurality of extensible markup language documents, the computer program instructions, which when executed on a processor, cause the processor to perform operations comprising:

traversing the plurality of extensible markup language documents by following each link of the plurality of extensible markup language documents;

parsing the plurality of extensible markup language documents to determine a structural hierarchy of the plurality of extensible markup language documents;

associating a task complexity with the plurality of extensible markup language documents based on the structural hierarchy, the associating comprising;

extracting a plurality of blocks of information from the plurality of extensible markup language documents; and

assigning a block of information in the plurality of blocks of information to a category in a plurality of categories, the plurality of categories comprising a task complexity category, the block of information comprising a value indicative of a number of extensible markup language documents associated with the plurality of extensible markup language documents and a value indicative of a number of links associated with the plurality of extensible markup language documents; and

characterizing the plurality of extensible markup language documents based on the task complexity.

12. The non-transitory computer readable medium of claim 11, the operations further comprising:

displaying the structural hierarchy of the plurality of extensible markup language documents.

13. The non-transitory computer readable medium of claim 11, the operations further comprising:

parsing the extensible markup language documents using document object model parsing.

14. The non-transitory computer readable medium of claim 11, the operations further comprising:

downloading the extensible markup language documents to a local disk.