PRODUCING WEB PAGE CONTENT

• US 20130124953A1
• Filed: 07/28/2010
• Published: 05/16/2013
• Est. Priority Date: 07/28/2010
• Status: Active Grant

• Current Assignee: Hewlett-Packard Development Company, L.P. (HP Inc.)
• Original Assignee: Hewlett-Packard Development Company, L.P. (HP Inc.)
• Inventors: Fan, Jian, Luo, Ping, Zheng, Li-Wei, Liu, Samson J., Lim, Suk Hwan, Xiong, Yuhong, Liu, Jerry J.
• Granted Patent: US 9,218,322 B2
• Time in Patent Office: Days
• US Class Current: 715/202
• CPC Class Codes: G06F 16/986 Document structures and storage, e.g. HTML extensions
• CPC Class Codes: G06F 40/10 Text processing natural language analysis G06F40/20; semantic analysis G06F40/30; processing or translation of natural language G06F40/40
• CPC Class Codes: G06F 40/106 Display of layout of documents; Previewing
• CPC Class Codes: G06F 40/154 Tree transformation for tree-structured or markup documents, e.g. XSLT, XSL-FO or stylesheets

Abstract

A method for producing web page content includes identifying blocks within a web page. The blocks are selectively assembled into sections. The sections are selectively assembled into article candidates. An article candidate that includes article content is distinguished from article candidates that do not include article content. Content is produced only from the article candidate distinguished as including article content.

Claims

1. A computer readable medium having processor executable instructions stored thereon, the instructions when executed cause the implementation of a method for producing web page content, the method comprising:

identifying blocks within a web page;

identifying blocks within a web page;

selectively assembling the blocks into sections;

selectively assembling the sections into article candidates;

distinguishing an article candidate that includes article content from article candidates that do not include article content; and

producing content only from the article candidate distinguished as including article content.

2. The medium of claim 1, wherein selectively assembling blocks into sections comprises:

identifying one or more frames within the web page, each frame encompassing one or more of the blocks; and

grouping the blocks into sections such that each section only includes blocks that are not separated by a frame.

3. The medium of claim 2, wherein the blocks include text blocks arranged in a top to bottom order within the web page and wherein grouping comprises:

combining a text block with a subsequent text block within the order only if the two visual text blocks are not separated by a frame, have no vertical overlap, share at least a partial horizontal overlap, and share a common visual attribute; and

following the top to bottom order, iteratively repeating the combining until no text blocks can be combined.

4. The medium of claim 1, wherein selectively assembling the sections into article candidates comprises, following a top to bottom order, combining a section with a previous section in the order upon a positive determination of one or more the following:

the section is a text section with a character count that exceeds a minimum threshold and has a left or right margin that is substantially aligned with the left or right margin of the previous section;

the section is a text section sharing visual attributes of the previous section; and

the section is an image section having a size exceeding a threshold size and shares at least a partial horizontal overlap with the previous section.

5. The medium of claim 1, wherein selectively assembling sections into article candidates comprises combining text and image sections separated by horizontal lines into article candidates, wherein combining includes:

detecting horizontal lines between sections; and

preventing two sections separated by a detected horizontal line from being combined if that horizontal line has a length that is substantially equal to a greater of the widths of two sections and either one of the two sections is a text section that does not have a character count that exceeds a threshold or one of the two sections is an image section whose size does not exceed a threshold size.

6. The medium of claim 1, wherein distinguishing article candidates includes identifying an article candidate that includes an article body, the identified article candidate being an article candidate that occupies a larger area near a top and center of the web page than other article candidates.

7. The medium of claim 6, wherein distinguishing article candidates includes identifying an article title, the article title being text that:

has a top that is positioned no lower than a top of the article body; and

has a text size that is no smaller than that of the article body.

8. The medium of claim 6, wherein distinguishing article candidates includes identifying an image caption, the image caption being text that is not identified as part of the article body and is positioned adjacent to an image that is part of the article body.

9. A system for producing web page content, the system comprising:

a block engine to identify blocks within a web page;

a assembly engine to selectively assemble the blocks into sections and to selectively assemble the sections into article candidates;

an article engine to distinguish an article candidate that includes article content from article candidates that do not include article content; and

a production engine to produce content only from the article candidate distinguished as including article content.

10. The system of claim 9, wherein the assembly engine selectively assembles blocks into sections by:

identifying one or more frames within the web page, each frame encompassing one or more of the blocks; and

grouping the blocks into sections such that each section only includes blocks that are not separated by a frame.

11. The system of claim 10, wherein the blocks include text blocks arranged in a top to bottom order within the web page and wherein the assembly engine groups the blocks by:

combining a text block with a subsequent text block within the order only if the to visual text blocks are not separated by a frame, have no vertical overlap, share at least a partial horizontal overlap, and share a common visual attribute; and

following the top to bottom order, iteratively repeating the combining until no text blocks can be combined.

12. The system of claim 9, wherein the assembly engine selectively assembles the sections into article candidates by, following a top to bottom order, combining a section with a previous section in the order upon a positive determination of one or more the following:

the section is a text section with a character count that exceeds a minimum threshold and has a left or right margin that is substantially aligned with the left or right margin of the previous section;

the section is a text section sharing visual attributes of the previous section; and

the section is an image section having a size exceeding a threshold size and shares at least a partial horizontal overlap with the previous section.

13. The system of claim 9, wherein selectively the assembly engine selectively assembles sections into article candidates by combining text and image sections separated by horizontal lines into article candidates, wherein combining includes:

detecting horizontal lines between sections; and

preventing two sections separated by a detected horizontal line from being combined if that horizontal line has a length that is substantially equal to a greater of the widths of two sections and either one of the two sections is a text section that does not have a character count that exceeds a threshold or one of the two sections is an image section whose size does not exceed a threshold size.

14. The system of claim 9, wherein the article engine distinguishes article candidates by identifying an article candidate that includes an article body, the identified article candidate being an article candidate that occupies a larger area near a top and center of the web page than other article candidates.

15. The system of claim 14, wherein the article engine distinguishes article candidate by identifying an article title, the article title being text that:

has a top that is positioned no lower than a top of the article body; and

has a text size that is no smaller than that of the article body.

16. The system of claim 14, wherein the article engine distinguishes article candidates by identifying an image caption, the image caption being text that is not identified as part of the article body and is positioned adjacent to an image that is part of the article body.

17. A method for producing web page content, the method comprising:

identifying blocks within a web page;

selectively assembling the blocks into sections;

selectively assembling the sections into article candidates;

distinguishing an article candidate that include article content from article candidates that do not include article content; and

producing content only from the article candidate distinguished as including article content.

18. The method of claim 17, wherein selectively assembling blocks into sections comprises:

identifying one or more frames within the web page, each frame encompassing one or more of the blocks; and

grouping the blocks into sections such that each section only includes blocks that are not separated by a frame.

19. The method of claim 17, wherein selectively assembling the sections into article candidates comprises, following a top to bottom order, combining a section with a previous section in the order upon a positive determination of one or more the following:

the section is a text section with a character count that exceeds a minimum threshold and has a left or right margin that is substantially aligned with the left or right margin of the previous section;

the section is a text section sharing visual attributes of the previous section; and

the section is an image section having a size exceeding a threshold size and shares at least a partial horizontal overlap with the previous section.

20. The method of claim 17, wherein distinguishing article candidates includes:

identifying an article candidate that includes an article body, the identified article candidate being an article candidate that occupies a larger area near a top and center of the web page than other article candidates;

identifying an article title, the article title being text that has a top that is positioned no lower than a top of the article body and has a text size that is no smaller than that of the article body; and

identifying an image caption, the image caption being text that is not identified as part of the article body and is positioned adjacent to an image that is part of the article body.