DOCUMENT OBJECT MODEL (DOM) BASED PAGE UNIQUENESS DETECTION

• US 20120005211A1
• Filed: 06/23/2011
• Published: 01/05/2012
• Est. Priority Date: 06/30/2010
• Status: Active Grant

• Current Assignee: Terrace Licensing LLC (IP Edge LLC)
• Original Assignee: International Business Machines Corporation
• Inventors: ALY, HOSAM, AYOUB, KHALIL, WALSH, JASON
• Granted Patent: US 8,489,605 B2
• Time in Patent Office: Days
• US Class Current: 707/737
• CPC Class Codes: G06F 16/81 Indexing, e.g. XML tags; Data structures therefor; Storage structures
• CPC Class Codes: G06F 16/951 Indexing; Web crawling techniques
• CPC Class Codes: G06F 16/958 Organisation or management of web site content, e.g. publishing, maintaining pages or automatic linking
• CPC Class Codes: G06F 3/0482 Interaction with lists of selectable items, e.g. menus

Abstract

DOM based unique ID generation, including receiving a hypertext markup language (HTML) page at a computer, and identifying HTML page elements in response to the receiving, the HTML page elements comprising parent nodes, the parent nodes comprising child nodes. The method further comprising processing each of the HTML page elements, the processing comprising: grouping the child nodes by parent node into a group of child nodes, detecting patterns in the group of child nodes in response to the grouping, reducing the group of child nodes to text strings in response to the detecting, storing the text strings as text values in the parent nodes, and generating a unique identifier (ID) of the HTML page in response to the processing.

Claims

1. A method comprising:

receiving a hypertext markup language (HTML) page at a computer;

identifying HTML page elements in response to the receiving, the HTML page elements comprising parent nodes, the parent nodes comprising child nodes;

processing each of the HTML page elements, the processing comprising;

grouping the child nodes by parent node into a group of child nodes;

detecting patterns in the group of child nodes in response to the grouping;

reducing the group of child nodes to text strings in response to the detecting; and

storing the text strings as text values in the parent nodes; and

generating a unique identifier (ID) of the HTML page in response to the processing.

2. The method of claim 1, wherein the processing further comprising:

sorting the group of child nodes in response to the reducing.

3. The method of claim 1, wherein the HTML page is a Web 2.0 page, the Web 2.0 page comprising content, the content being generated dynamically.

4. The method of claim 2, wherein the HTML page comprises a visible page, the visible page comprising moveable HTML page elements, the moveable HTML page elements configured to occupy any location on the visible page, and the unique ID of the HTML page is the same for all locations of the moveable HTML page elements on the visible page.

5. The method of claim 1, wherein the HTML page comprises a uniform resource locator (URL), and the unique ID of the HTML page is the same for a plurality of HTML pages, the plurality of HTML pages each comprising a different URL.

6. The method of claim 1, further comprising filtering HTML page elements in response to the identifying, the filtering removing the child nodes and the parent nodes that meet filter criteria, the filter criteria comprising on or more of:

extensible markup language path language instructions;

regular expression instructions; and

a list of html nodes.

7. A system comprising:

a host system in communication with at least one client system over a network;

a page based unique ID generation application for execution on the host system, the page based for unique ID generation application including logic for implementing a method comprising;

receiving a hypertext markup language (HTML) page at a computer;

identifying HTML page elements in response to the receiving, the HTML page elements comprising parent nodes, the parent nodes comprising child nodes;

processing each of the HTML page elements, the processing comprising;

grouping the child nodes by parent node into a group of child nodes;

detecting patterns in the group of child nodes in response to the grouping;

reducing the group of child nodes to text strings in response to the detecting; and

storing the text strings as text values in the parent nodes; and

generating a unique identifier (ID) of the HTML page in response to the processing.

8. The system of claim 7, wherein the processing further comprising:

sorting the group of child nodes in response to the reducing.

9. The system of claim 7, wherein the HTML page is a Web 2.0 page, the Web 2.0 page comprising content, the content being generated dynamically.

10. The system of claim 8, wherein the HTML page comprises a visible page, the visible page comprising moveable HTML page elements, the moveable HTML page elements configured to occupy any location on the visible page, and the unique ID of the HTML page is the same for all locations of the moveable HTML page elements on the visible page.

11. The system of claim 7, wherein the HTML page comprises a uniform resource locator (URL), and the unique ID of the HTML page is the same for a plurality of HTML pages, the plurality of HTML pages each comprising a different URL.

12. The system of claim 7, further comprising filtering HTML page elements in response to the identifying, the filtering removing the child nodes and the parent nodes that meet filter criteria, the filter criteria comprising on or more of:

extensible markup language path language instructions;

regex instructions; and

a list of html nodes.

13. A computer program product comprising a non-transitory storage medium storing instructions, the computer program product implementing a method, the method comprising:

receiving a hypertext markup language (HTML) page at a computer;

identifying HTML page elements in response to the receiving, the HTML page elements comprising parent nodes, the parent nodes comprising child nodes;

processing each of the HTML page elements, the processing comprising;

grouping the child nodes by parent node into a group of child nodes;

detecting patterns in the group of child nodes in response to the grouping;

reducing the group of child nodes to text strings in response to the detecting; and

storing the text strings as text values in the parent nodes; and

generating a unique identifier (ID) of the HTML page in response to the processing.

14. The computer program product of claim 13, wherein the processing further comprising:

sorting the group of child nodes in response to the reducing.

15. The computer program product of claim 13, wherein the HTML page is a Web 2.0 page, the Web 2.0 page comprising content, the content being generated dynamically.

16. The computer program product of claim 14, wherein the HTML page comprises a visible page, the visible page comprising moveable HTML page elements, the moveable HTML page elements configured to occupy any location on the visible page, and the unique ID of the HTML page is the same for all locations of the moveable HTML page elements on the visible page.

17. The computer program product of claim 13, wherein the HTML page comprises a uniform resource locator (URL), and the unique ID of the HTML page is the same for a plurality of HTML pages, the plurality of HTML pages each comprising a different URL.

18. The computer program product of claim 13, further comprising filtering HTML page elements in response to the identifying, the filtering removing the child nodes and the parent nodes that meet filter criteria, the filter criteria comprising on or more of:

extensible markup language path language instructions;

regex instructions; and

a list of html nodes.

19. An apparatus comprising:

web indexing application logic communicatively coupled to a computer processor and configured to receive a hypertext markup language (HTML) page at a computer, identify HTML page elements in response to the receiving, the HTML page elements comprising parent nodes, the parent nodes comprising child nodes, and process each of the HTML page elements, the processing comprising;

grouping the child nodes by parent node into a group of child nodes;

detecting patterns in the group of child nodes in response to the grouping;

reducing the group of child nodes to text strings in response to the detecting; and

storing the text strings as text values in the parent nodes; and

the web indexing application logic further configured to generate a unique identifier (ID) of the HTML page in response to the processing.

20. The apparatus of claim 19, wherein the processing further comprising:

sorting the group of child nodes in response to the reducing.

21. The apparatus of claim 19, wherein the HTML page is a Web 2.0 page, the Web 2.0 page comprising content, the content being generated dynamically.

22. The apparatus of claim 20, wherein the HTML page comprises a visible page, the visible page comprising moveable HTML page elements, the moveable HTML page elements configured to occupy any location on the visible page, and the web indexing application logic configured to generate the same unique ID of the HTML page for all locations of the moveable HTML page elements on the visible page.

23. The apparatus of claim 19, wherein the HTML page comprises a uniform resource locator (URL), and the unique ID of the HTML page is the same for a plurality of HTML pages, the plurality of HTML pages each comprising a different URL.

24. The apparatus of claim 19, wherein the web indexing application logic is further configured to filter HTML page elements in response to the identifying, the filtering removing the child nodes and the parent nodes that meet filter criteria, the filter criteria comprising on or more of:

extensible markup language path language instructions;

regular expression instructions; and

a list of html nodes.

25. A computing system comprising:

at least one processor;

at least one memory;

a network transceiver;

a bus communicatively linking said at least one processor, memory, and network transceiver to each other, wherein said at least one memory stores an executable instructions, which the at least one processor executes,

wherein said network transceiver of the computing system comprising hardware is operable to receive a hypertext markup language (HTML) page from a server over a network, wherein the computing system is configured to function as a client in a client-server arrangement with the server,

wherein the processor executing the instructions is operable to identify HTML page elements in response to the receiving of the HTML page, the HTML page elements comprising parent nodes, the parent nodes comprising child nodes;

wherein the processor executing the instructions is operable to process each of the HTML page elements, the processing comprising;

grouping the child nodes by parent node into a group of child nodes;

detecting patterns in the group of child nodes in response to the grouping;

reducing the group of child nodes to text strings in response to the detecting; and

storing the text strings as text values in the parent nodes; and

wherein the processor executing the instructions is operable to generate a unique identifier (ID) of the HTML page in response to the processing.