Visual web page analysis system and method

• US 10,664,647 B2
• Filed: 10/28/2014
• Issued: 05/26/2020
• Est. Priority Date: 10/28/2014
• Status: Active Grant

• Current Assignee: Kuang-Yao Lee
• Original Assignee: Kuang-Yao Lee
• Inventors: Lee, Kuang-Yao
• Primary Examiner(s): Khan, Shahid K
• Application Number:
  US14/525,282
• Publication Number:
  US 20160117292A1
• Time in Patent Office: 2,037 Days
• CPC Class Codes: G06F 16/9577 Optimising the visualization of content, e.g. distillation of HTML documents
• CPC Class Codes: G06F 40/143 Markup, e.g. Standard Generalized Markup Language [SGML] or Document Type Definition [DTD]
• CPC Class Codes: G06T 2207/30 Subject of image; Context of image processing
• CPC Class Codes: G06V 30/416 Extracting the logical structure, e.g. chapters, sections or page numbers; Identifying elements of the document, e.g. authors

Abstract

A visual web page analysis system includes an image analyzing unit, a block analyzing unit, a vision identifying unit, and an output unit. The image analyzing unit loads information of a web page and segments content of the web page into a plurality of blocks based on a visual feature. The block analyzing unit classifies the blocks based on an attribute of each block. The vision identifying unit compares at least a relative feature of each block to determine a function of each block on the web page. The output unit collects the blocks and their functions into an information interface and outputs the information interface.

Claims

1. A visual web page analysis system for analyzing data of a web page based on vision, the system comprising:

a processor;

an image analyzing program executed by a processer to enable a processer to load information of a web page and to segment content of the web page into a plurality of blocks based on at least a visual feature of the web page;

a block analyzing program executed by the processer to enable the processer to classify the plurality of blocks based on at least an attribute of each block;

a vision identifying program executed by the processer to enable the processer to compare at least a relative feature of each block to determine a function of each classified block on the web page; and

an output program executed by the processer to enable the processer to collect the plurality of blocks and their functions into an information interface and to output the information interface,

wherein the processor provides an analyzed result shown on the information interface, and

wherein the block analyzing program executed by the processer further enables the processer to receive a plurality of web page tags, to determine the attribute of each block in accordance with the following formulas for Degree of Picture Hyperlink (DoPH), Picture Text Ratio (PTR), Degree of Local Text Hyperlink (DoLTH), Text Ratio, and Degree of Local Picture Hyperlink (DoLPH);

(DoTH)=(number of text hyperlinks in the block)/(number of text tags in the block), where a text tag is any HTML grammar instruction that can be used to present texts;

(DoPH)=(number of picture hyperlinks in the block)/(number of picture tags in the block), where a picture tag is any HTML grammar instruction that can be used to present pictures;

Text Ratio=(number of characters in the block)/(number of characters in the web page);

(PTR)=(number of image tags in the block)/(number of text tags in the block), where the PTR is used to measure a pictures-versus-text ratio in the block;

(DoLTH)=(number of local text hyperlinks in the block)/(number of text hyperlinks in the block), where the local text hyperlinks are text hyperlinks linked to the same web domain; and

(DoLPH)=(number of local picture hyperlinks in the block)/(number of picture hyperlinks in the block), where the local picture hyperlinks are picture hyperlinks linked to the same web domain.

2. The visual web page analysis system as claimed in claim 1, wherein the image analyzing program includes an image processing program executed by the processer to enable the processer to perform an image processing to distinguish texts and pictures from the web page.

3. The visual web page analysis system as claimed in claim 1, wherein the image analyzing program further includes an image segment and composition program executed by the processer to enable the processer to compose texts and pictures with a related visual feature into a block based on at least one visual feature.

4. The visual web page analysis system as claimed in claim 2, wherein the image analyzing unit further includes a block checking program executed by the processor to enable the processor to use a position of an auxiliary object to check whether the plurality of blocks are segmented correctly or not.

5. The visual web page analysis system as claimed in claim 4, wherein the auxiliary object is a separator line on the web page.

6. The visual web page analysis system as claimed in claim 1, wherein the block analyzing unit program enables the processer to determine the attribute of each block in accordance with HTML codes of the block.

7. The visual web page analysis system as claimed in claim 1, wherein the block analyzing unit program enables the processer to determine whether the attribute of each block is input box or dropdown menu in accordance with a visual characteristic of the block.

8. The visual web page analysis system as claimed in claim 1, wherein the relative feature includes relative positions of the plurality of blocks.

9. The visual web page analysis system as claimed in claim 8, wherein the relative feature further includes sizes of the plurality of blocks.

10. The visual web page analysis system as claimed in claim 1, wherein the relative feature further includes layouts of objects of the plurality of the blocks.

11. The visual web page analysis system as claimed in claim 1, wherein the relative feature further includes visual characteristics of the objects of the blocks.

14. The visual web page analysis method as claimed in claim 11, wherein step (b) further includes:

composing texts and pictures with a related vision feature into a block.

12. A visual web page analysis method executed by a processor for analyzing data of a web page based on vision, comprising the steps of:

(a) loading information of a web page;

(b) segmenting content of the web page into a plurality of blocks based on at least a visual feature of the web page;

(c) classifying the plurality of blocks based on at least an attribute of each block;

(d) comparing at least a relative feature of each block to determine a function of each classified block on the web page; and

(e) collecting the plurality of blocks and their functions into an information interface, and outputting the information interface,

wherein step (d) further includes;

receiving a plurality of web page tags, to determine the attribute of each block in accordance with the following formulas for Degree of Picture Hyperlink (DoPH), Picture Text Ratio (PTR), Degree of Local Text Hyperlink (DoLTH), Text Ratio, and Degree of Local Picture Hyperlink (DoLPH);

(DoTH)=(number of text hyperlinks in the block)/(number of text tags in the block), where a text tag is any HTML grammar instruction that can be used to present texts;

(DoPH)=(number of picture hyperlinks in the block)/(number of picture tags in the block), where a picture tag is any HTML grammar instruction that can be used to present pictures;

Text Ratio=(number of characters in the block)/(number of characters in the web page);

(PTR)=(number of image tags in the block)/(number of text tags in the block), where the PTR is used to measure a pictures-versus-texts ratio in the block;

(DoLTH)=(number of local text hyperlinks in the block)/(number of text hyperlinks in the block), where the local text hyperlinks are text hyperlinks linked to the same web domain; and

(DoLPH)=(number of local picture hyperlinks in the block)/(number of picture hyperlinks in the block), where the local picture hyperlinks are picture hyperlinks linked to the same web domain.

13. The visual web page analysis method as claimed in claim 12, wherein step (b) further includes:

performing an image processing to distinguish texts and pictures from the web page.

15. The visual web page analysis method as claimed in claim 12, wherein the step (b) further includes:

using a position of an auxiliary object to check whether the plurality of blocks are segmented correctly or not.

16. The visual web page analysis method as claimed in claim 15, wherein the auxiliary object is a separator line on the web page.

17. The visual web page analysis method as claimed in claim 12, wherein the relative feature in step (d) includes relative positions of the plurality of blocks.

18. The visual web page analysis method as claimed in claim 12, wherein the relative feature further includes sizes of the plurality of blocks.

19. The visual web page analysis method as claimed in claim 12, wherein the information interface is a picture.

20. The visual web page analysis method as claimed in claim 12, wherein the information interface is a web page.