Hosted searching of private local area network information with support for add-on applications

• US 7,877,368 B2
• Filed: 11/02/2007
• Issued: 01/25/2011
• Est. Priority Date: 11/02/2007
• Status: Active Grant

• Current Assignee: Citrix Systems, Inc. (Cloud Software Group)
• Original Assignee: Paglo Labs, Inc. (Cloud Software Group)
• Inventors: Waters, Christopher, Lockhart, Andrew, de Haaff, Brian
• Primary Examiner(s): Breene; John E
• Assistant Examiner(s): Nguyen; Phong
• Application Number:
  US11/982,495
• Publication Number:
  US 20090119280A1
• Time in Patent Office: 1,180 Days
• Field of Search: 707/999.003, 707/999.005, 707/999.102, 707/706, 707/802, 709/203, 709/223, 709/224
• US Class Current: 707/706
• CPC Class Codes: G06F 16/951 Indexing; Web crawling techniques
• CPC Class Codes: H04W 4/60 Subscription-based services using application servers or record carriers, e.g. SIM application toolkits

Abstract

Hosted searching of private LAN information is described. The apparatus includes a LAN crawler to automatically and repeatedly crawl a LAN having multiple devices, using a discovery module to discover the devices, a generic-probing module to attempt to collect the descriptive information according to a first set of probing requirements, and multiple specific-probing plug-ins each of which attempt to collect the descriptive information according to a second set of specific probing requirements. In another embodiment, the apparatus also includes a hosted on-demand search system including a centralized-search server to create and synchronize a private search database. The centralized-search server includes an application interface to receive a request to access the private search database from a third-party add-on application, to provide the accessed information to the third-party add-on application, and to receive from the third-party add-on application an application rendered component to be displayed on the user interface.

Claims

1. An apparatus for hosted searching of private local area network (LAN) information comprising;

a LAN crawler to automatically and repeatedly crawl a LAN having a plurality of devices, said LAN crawler including,

a collection module to attempt to collect, from across each of a plurality of disparate information sources stored in the plurality of devices, only those parts of the disparate information sources that include descriptive information of a first plurality of items of interest, wherein for at least one of the plurality items of interest, the parts include technical information regarding the private LAN, users of the private LAN, and the plurality of devices of the private LAN, and

a reporting module to automatically report the collected information organized according to the plurality of items out of a firewall of the private LAN being crawled and over the Internet; and

a hosted on-demand search system to be coupled to the Internet including, a set of one or more centralized-search servers including,

a crawler interface to receive the reports from the LAN crawler deployed on the private LAN,

a database interface to create and synchronize a private search database for the private LAN based on the received reports, wherein the private search database includes the collected information collected from across the disparate information sources of that private LAN organized by the first plurality of items of interest, and wherein the private search database is not organized by documents located on the plurality of devices of the private LAN,

a server user interface module to provide a user interface over the Internet to the private LAN, to perform private searches of the private search database, and to provide an application rendered component as part of the user interface, and

an application interface to receive a request to access the collected information stored in the private search database from a third-party add-on application, to provide the accessed information to the third-party add-on application, and to receive from the third-party add-on application the application rendered component to be displayed on the user interface;

wherein the plurality of disparate information sources include a routing table of a router, a router configuration file of a router, an address resolution protocol (ARP) table of a switch, a bridging table of a switch, a switch configuration table of a switch, a software configuration file of a server, a resource utilization counter of a server, a log file of a server, custom application file of a server, user information of a directory server, group information of a directory server, a software configuration file of a workstation, a registry of a workstation, an operating system interface database of a workstation, a directory of a workstation, a resource utilization counter of a workstation, a log file of a workstation, and a configuration file of a network printer.

2. The apparatus of claim 1, wherein the third-party add-on application is executed by a remote server and the third-party add-on application communicates with the server user interface module through the application interface.

3. The apparatus of claim 1, wherein the LAN crawler is deployed on one of the plurality of devices in each of the plurality of different private LANS, the LAN crawler to collect values of the technical information from the plurality of disparate information sources stored on the plurality of devices of the LAN being crawled through a set of one or more organized questions to the plurality of devices of the private LAN being crawled.

4. The apparatus of claim 1, wherein the set of one or more centralized-search servers further includes a configuration database coupled to the server user interface module to store authentication information for access by the third-party add-on application and search queries.

5. The apparatus of claim 1, wherein the server user interface module comprises:

an application rendering module to render an application rendered component to be displayed in the user interface, wherein the application rendered component represents the search results of the private search by the third-party add-on application; and

a user interface rendering module to generate a user interface page including the application rendered component to be displayed in the user interface.

6. The apparatus of claim 1, wherein the LAN crawler further includes an organization module to automatically organize the collected information according to the first plurality of items of interest.

7. The apparatus of claim 1, wherein the collection module is configured to attempt to collect the technical information from the plurality of disparate information sources through a set of one or more organized questions to the plurality of devices of the private LAN being crawled, the collection module receiving values in response to the set of organized questions from the plurality of disparate information sources, wherein the values, received in response to the set of organized questions, include the descriptive information of the first plurality of items of interest.

8. The apparatus of claim 1, wherein at least some of the private search databases are each a hierarchical database, wherein the hierarchical database includes one or more sub-trees branching from a tree root node, wherein each sub-tree includes one or more nodes starting at a sub-tree root node, and wherein each node has a unique identifier, and wherein each hierarchical database is searched with a search query that includes a SELECT clause and a FROM clause, wherein the SELECT clause includes syntax to identify a path in the hierarchical database starting at a root of the hierarchy, and wherein the FROM clause describes from where in the hierarchical database information should be returned.

9. The apparatus of claim 8, wherein the hierarchical database is extended for the third-party add-on application by creating a new sub-tree from the tree root node that corresponds to that third-party add-on application'"'"'s application provider, wherein information in that new sub-tree can be accessed only through that third-party add-on application.

10. The apparatus of claim 1, wherein the set of centralized-search servers comprises a reorganization module to reorganize the collected information received in the reports, the reorganization module including,

a report scanning module to attempt to collect, from across the reports, descriptive information of a second plurality of items of interest, wherein the descriptive information of the second plurality of items of interest include LAN-centric network topology information;

and

an integration module to integrate the collected descriptive information of the second plurality of items of interest organized according to the second plurality of items of interest into the corresponding private search database.

11. The apparatus of claim 10, wherein the third-party add-on application is configured to send the search request to the application interface to perform the private search of the private search database to extract the LAN-centric network topology information for the private LAN, create a network topology map to be displayed on the user interface based on the LAN-centric network topology information, and to send the network topology map to the server user interface module to provide the network topology map as part of the application rendered component in the user interface.

12. The apparatus of claim 1, wherein the third-party add-on application is configured to submit additional data to the application interface to be stored in the private search database.

13. The apparatus of claim 1, wherein the hosted on-demand search system further comprises a classification server coupled to a classification database and the set of centralized-search servers, the classification server to compare the collected information of the private search databases against predetermined characteristics of known devices, and to compute a classification metric for each of the plurality of devices of the private LAN.

14. A method for hosted searching of private local area network (LAN) information, comprising:

automatically and repeatedly crawling a private LAN having a plurality of devices to collect, from across each of a plurality of disparate information sources stored in the plurality of devices of the private LAN, only those parts of the disparate information sources that include descriptive information of a first plurality of items of interest, wherein for at least one of the plurality items of interest, the parts include technical information regarding the private LAN, users of the private LAN, and the plurality of devices of the private LAN;

automatically reporting the collected information organized according to the first plurality of items of interest out of a of the private LAN being crawled and over the Internet to a hosted on-demand search system;

receiving the reports from the private LAN being crawled at the hosted demand search system;

creating and synchronizing a private search database for the private LAN based on the received reports, wherein the private search database includes the descriptive information collected from across the disparate information sources of that private LAN organized by the first plurality of items of interest, and wherein the private search database is not organized by documents located on the private LAN;

receiving a request to access the collected information stored in the private search database from a third-party add-on application; and

providing the accessed information to the third-party add-on application;

wherein the plurality of disparate information sources include a routing table of a router, a router configuration file of a router, an address resolution protocol (ARP) table of a switch, a bridging table of a switch, a switch configuration table of a switch, a software configuration file of a server, a resource utilization counter of a server, a log file of a server, custom application file of a server, user information of a directory server, group information of a directory server, a software configuration file of a workstation, a registry of a workstation, an operating system interface database of a workstation, a directory of a workstation, a resource utilization counter of a workstation, a log file of a workstation, and a configuration file of a network printer.

15. The method of claim 14, wherein said automatically and repeatedly crawling that private LAN to collect descriptive information comprises;

asking a set of one or more organized questions of the plurality of disparate information sources stored in the plurality of devices of the private LAN being crawled; and

receiving values of the technical information from the plurality of disparate information sources in response to the set of organized questions.

16. The method of claim 14, further comprising:

receiving from the third-party add-on application an application rendered component to be displayed on the user interface; and

providing a user interface over the Internet to the private LAN that includes the application rendered component.

17. The method of claim 14, further comprising:

storing a configuration of the application rendered component, including the search query, in a configuration database for later retrieval; and

allowing display and selection of the stored configuration of the application rendered component, but not the search results of the private search by the third party add-on application, through the user interface when the stored search query is designated as shareable.

18. The method of claim 14, further comprising:

organizing the collected information according to the first plurality of items of interest before reporting the collected information out of the firewall; and

integrating the organized information into the corresponding private search databases.

19. The method of claim 14, further comprising:

collecting from across the reports descriptive information of a second plurality of items of interest;

organizing the collected descriptive information of the second plurality of items of interest according to the second plurality of items of interest; and

integrating the collected descriptive information of the second plurality of items of interest organized according to the second plurality of items of interest into the corresponding private search database reorganized information into the corresponding private search databases.

20. The of claim 14, wherein automatically and repeatedly crawling the private LAN to collect descriptive information comprises:

asking a set of one or more organized questions to the plurality of devices of the private LAN being crawled; and

receiving values in response to the set of organized questions from the plurality of disparate information sources, wherein the received values are the descriptive information of the plurality of items of interest.

21. The method of claim 14, further comprising:

comparing the collected information of the private search database against predetermined characteristics of known devices; and

computing a classification metric for each of the plurality of devices of the private LAN.

22. A method for hosted searching of private local area network (LAN) information, comprising:

automatically and repeatedly crawling a private LAN having a plurality of devices to collect, from across each of a plurality of disparate information sources stored in the plurality of devices of the private LAN, only those parts of the disparate information sources that include descriptive information of a first plurality of items of interest, wherein for at least one of the plurality items of interest, the parts include technical information regarding the private LAN, users of the private LAN, and the plurality of devices of the private LAN;

automatically reporting the collected information organized according to the first plurality of items of interest out of a of the private LAN being crawled and over the Internet to a hosted on-demand search system;

receiving the reports from the private LAN being crawled at the hosted demand search system;

creating and synchronizing a private search database for the private LAN based on the received reports, wherein the private search database includes the descriptive information collected from across the disparate information sources of that private LAN organized by the first plurality of items of interest, and wherein the private search database is not organized by documents located on the private LAN;

receiving a request to access the collected information stored in the private search database from a third-party add-on application;

providing the accessed information to the third-party add-on application;

receiving at the hosted on-demand search system a first request from the private LAN to display the user interface with the application rendered component;

sending a second request to the third-party add-on application to generate the application rendered component;

receiving from the third-party add-on application a third request to perform one or more private searches for the descriptive information of the first plurality of items of interest in the private search database;

authorizing one or more private searches using a configuration database;

sending search results of the one or more private searches to the third-party add-on application;

generating the application rendered component to represent the search results of the one or more private searches;

generating a page for the user interface that includes the application rendered component that represent the search results of the one or more private searches; and

sending the generated page, including the application rendered component, to the user interface to be displayed.

23. A method for hosted searching of private local area network (LAN) information, comprising:

automatically and repeatedly crawling a private LAN having a plurality of devices to collect, from across each of a plurality of disparate information sources stored in the plurality of devices of the private LAN, only those parts of the disparate information sources that include descriptive information of a first plurality of items of interest, wherein for at least one of the plurality items of interest, the parts include technical information regarding the private LAN, users of the private LAN and the plurality of devices of the private LAN;

automatically reporting the collected information organized according to the first plurality of items of interest out of a of the private LAN being crawled and over the Internet to a hosted on-demand search system;

receiving the reports from the private LAN being, crawled at tile hosted demand search system;

creating and synchronizing a private search database for the private LAN based on the received reports, wherein the private search database includes the descriptive information collected from across the disparate information sources of that private LAN organized by the first plurality, of items of interest, and wherein the private search database is not organized by documents located on the private LAN;

receiving a request to access the collected information stored in the private search database from a third-party add-on application;

providing the accessed information to the third-party add-on application;

collecting from across the reports descriptive information of a second plurality of items of interest;

organizing the collected descriptive information of the second plurality of items of interest according to the second plurality of items of interest; and

integrating the collected descriptive information of the second plurality of items of interest organized according to the second plurality of items of interest into the corresponding private search database reorganized information into the corresponding private search databases;

wherein the descriptive information of the second plurality of items of interest includes LAN-centric network topology information, wherein the method further comprises;

receiving at the hosted on-demand search system a first request from the private LAN to display a network topology map to be displayed on the user interface as part of the application rendered component;

sending a second request to the third-party add-on application to generate the application rendered component that includes the network topology map;

receiving from the third-party add-on application a third request to perform one or more private searches for LAN-centric network topology information in the private search database;

authorizing the one or more private searches using a configuration database;

sending search results of the one or more private searches to the third-party add-on application;

generating the network topology map as part of the application rendered component based on the search results of the one or more private searches;

generating a page for the user interface that includes the application rendered component and the network topology map; and

sending the generated page, including the network topology map, to the user interface to be displayed.

24. An apparatus for hosted searching of private local area network (LAN) information comprising:

a LAN crawler to automatically and repeatedly crawl a LAN having a plurality of devices, said LAN crawler including,

a collection module to attempt to collect, from across each of a plurality of disparate information sources stored in the plurality of devices, only those parts of the disparate information sources that include descriptive information of a first plurality of items of interest, wherein for at least one of the first plurality items of interest, the parts are collected from multiple ones of the plurality of disparate information sources, wherein the collection module comprises;

a discovery module to discover the plurality of devices of the private LAN;

a generic-probing module to attempt to collect the descriptive information from the disparate information sources stored on the plurality of devices of the private LAN according to a first set of probing requirements; and

a plurality of specific-probing plug-ins that are coupled to the collection module to attempt to collect the descriptive information from the disparate information sources stored on the plurality of devices of the private LAN, each of the plurality of specific-probing plug-ins to attempt to collect the descriptive information according to a second set of probing requirements that are specific to each of the plurality of specific-probing plug-ins, wherein at least one of the plurality of specific-probing plug-ins is configured to attempt to collect the descriptive information from the disparate information sources stored on the plurality of devices of the private LAN according to one or more probing requirements that are specific to an add-on application; and

a reporting module to automatically report the collected information organized by the plurality of items of interest out of a firewall of the private LAN being crawled and over the Internet;

and

a hosted on-demand search system to be coupled to the Internet including, a set of one or more centralized-search servers including,

a crawler interface to receive the reports from the LAN crawler deployed on the private LAN,

a database interface to create and synchronize a private search database for the private LAN based on the received reports, wherein the private search database includes the collected information collected from across the disparate information sources of that private LAN organized by the first plurality of items of interest, and wherein the private search database is not organized by documents located on the plurality of devices of the private LAN,

a server user interface module to provide a user interface over the Internet to the private LAN, to perform private searches of the private search database, and to provide an application rendered component as part of the user interface, and

an application interface to receive a search request to perform a private search of the private search database from the add-on application, to provide search results of the private search to the add-on application, and to receive from the add-on application the application rendered component, representing the search results, to be displayed on the user interface;

wherein the plurality of disparate information sources include a routing table of a router, a router configuration file of a router, an address resolution protocol (ARP) table of a switch, a bridging table of a switch, a switch configuration table of a switch, a software configuration file of a server, a resource utilization counter of a server, a log file of a server, custom application file of a server, user of a directory server, group information of a directory server, a configuration file of a workstation, a registry of a workstation, an operating system interface database of a workstation, a directory of a workstation, a resource utilization counter of a workstation, a log file of a workstation, and a configuration file of a network printer.

25. The apparatus of claim 24, wherein the add-on application is configured to submit additional data to the application interface to be stored in the private search database.

26. The apparatus of claim 24, wherein the set of one or more centralized-search servers further includes a configuration database coupled to the server user interface module to store authentication information for access by the add-on application and search queries.

28. The apparatus of claim 24, wherein the collection module is configured to attempt to collect the technical information from the plurality of disparate information sources through a set of one or more organized questions to the plurality of devices of the private LAN being crawled, the collection module receiving values in response to the set of organized questions from the plurality of disparate information sources, and wherein the values, received in response to the set of organized questions, include the descriptive information of the first plurality of items of interest.

29. The apparatus of claim 24, wherein at least some of the private search databases are each a hierarchical database, wherein the hierarchical database includes one or more sub-trees branching a tree root node, wherein each sub-tree includes one or more nodes starting at a sub-tree root node, and wherein each node has a unique identifier, and wherein each hierarchical database is searched with a search query that includes a SELECT clause and a FROM clause, wherein the SELECT clause includes syntax to identify a path in the hierarchical database starting at a root of the hierarchy, and wherein the FROM clause describes from where in the hierarchical database information should be returned.

30. The apparatus of claim 29, wherein the hierarchical database is extended for the third-party add-on application by creating a new sub-tree from the tree root node that corresponds to that third-party add-on application'"'"'s application provider, wherein information in that new sub-tree can be accessed only through that third-party add-on application.

31. The apparatus of claim 24, wherein the hosted on-demand search system further comprises a classification server coupled to a classification database and the set of one or more centralized-search servers, the classification server to compare the collected information of the private search database against predetermined characteristics of known devices, and to compute a classification metric for each of the plurality of devices of the private LAN.

32. The apparatus of claim 24, wherein the descriptive information of the plurality of items of interest includes technical information regarding the private LAN, users of the private LAN, and the plurality of devices of the private LAN.

33. The apparatus of claim 24, wherein at least one of the plurality of specific-probing plug-ins is a periodic plug-in that attempts to collect the descriptive information from the plurality of devices of the private LAN periodically.

34. The apparatus of claim 24, wherein at least one of the plurality of specific-probing plug-ins is a device plug-in that attempts to collect the descriptive information from the plurality of devices of the private LAN that are a specified type of device according to a set of one or more probing requirements that are specific to the specified type of device.

35. The apparatus of claim 24, wherein at least one of the plurality of specific-probing plug-ins is a hybrid plug-in that operates as either:

a periodic plug-in that that attempts to collect the descriptive information at least one of the plurality of devices of the private LAN periodically;

or

a device plug-in that attempts to collect the descriptive information from the at least one device of the plurality of devices of the private LAN according to specific probing requirements of a specified type of information source.

27. The apparatus of 24, wherein the LAN crawler includes an organization module to automatically organize the collected information according to the first plurality of items of interest, wherein the plurality of items of interest include LAN-centric network topology information, and wherein the plurality of disparate information sources include a routing table.

36. A method for hosted searching of private local area network (LAN) information, comprising:

automatically and repeatedly crawling a private LAN having a plurality of devices to collect, from across each of a plurality of disparate information sources stored in the plurality of devices of the private LAN, only those parts of the disparate information sources that include descriptive information of a first plurality of items of interest, wherein for at least one of the plurality items of interest, wherein for at least one of the plurality items of interest, the parts are collected from multiple ones of the plurality of disparate information sources, wherein said automatically and repeatedly crawling comprises;

discovering the plurality of devices of the private LAN;

collecting the descriptive information from the disparate information sources stored on the plurality of devices of the private LAN according to a first set of probing requirements; and

collecting the descriptive information according to a second set of probing requirements, wherein said collecting the descriptive information according to the second set comprises collecting the descriptive information according to one or more probing requirements that are specific to an add-on application;

automatically reporting the collected information organized according to the first plurality of items of interest out of a firewall of the private LAN being crawled and over the Internet to a hosted on-demand search system;

receiving the reports from the private LAN being crawled at the hosted demand search system;

creating and synchronizing a private search database for the private LAN based on the received reports, wherein the private search database includes the descriptive information collected from across the disparate information sources of that private LAN organized by the first plurality of items of interest, and wherein the private search database is not organized by documents located on the private LAN;

receiving a search request to perform a private search of the private search database from the add-on application; and

providing search results of the private search to the add-on application;

wherein the descriptive information of the plurality of items of interest include technical information regarding the plurality of private LANs, users of the plurality of private LANs, and the plurality of devices of the plurality of private LANs;

wherein the plurality of disparate information sources include a routing table of a router, a router configuration file of a router, an address resolution protocol (ARP) table of a switch, a bridging table of a switch, a switch configuration table of a switch, a software configuration file of a server, a resource utilization counter of a server, a log file of a server, custom application file of a server, user information of a directory server, group information of a directory server, a software configuration file of a workstation, a registry of a workstation, an operating system interface database of a workstation, a directory of a workstation, a resource utilization counter of a workstation, a log file of a workstation, and a configuration file of a network printer.

37. The method of claim 36, wherein said automatically and repeatedly crawling that private LAN to collect descriptive information comprises:

asking a set of one or more organized questions of the plurality of disparate information sources stored in the plurality of devices of the private LAN being crawled; and

receiving values of the technical information from the plurality of disparate information sources in response to the set of organized questions.

38. The method of claim 36, comprising:

receiving from the add-on application an application rendered component that represents the search results to be displayed on the user interface; and

providing a user interface over the Internet to the private LAN that includes the application rendered component.

39. The method of claim 36, wherein said collecting the descriptive information according to the second set of probing requirements comprises collecting the descriptive information from at least one of the plurality of devices of the private LAN periodically.

40. The method of claim 36, further comprising organizing the collected descriptive information of the second plurality of items of interest according to the plurality of items of interest, wherein the first plurality of items of interest include LAN-centric network topology information, and wherein the plurality of disparate information sources include a routing table.

41. The method of claim 36, wherein said collecting the descriptive information according to the second set of probing requirements comprises collecting the descriptive information from the plurality of devices of the private LAN according to a set of one or more probing requirements that are specific to a specified type of information source.

42. The method of claim 36, further comprising:

collecting from across the reports descriptive information of a second plurality of items of interest;

organizing the collected descriptive information of the second plurality of items of interest according to the second plurality of items of interest; and

integrating the collected descriptive information of the second plurality of items of interest organized according to the second plurality of items of interest into the corresponding private search database reorganized information into the corresponding private search databases.

43. The method of claim 36, further comprising:

comparing the collected information of the private search database against predetermined characteristics of known devices; and

computing a classification metric for each of the plurality of devices of the private LAN.

44. A method for hosted searching of private local area network (LAN) information, comprising:

automatically and repeatedly crawling a private LAN having a plurality of devices to collect, from across each of a plurality of disparate information sources stored in the plurality of devices of the private LAN, only those parts of the disparate information sources that include descriptive information of a first plurality of items of interest, wherein for at least one of the plurality items of interest, wherein for at least one of the plurality items of interest, the parts are collected from multiple ones of the plurality of disparate information sources, wherein said automatically and repeatedly crawling comprises;

discovering the plurality of devices of the private LAN;

collecting the descriptive information from the disparate information sources stored on the plurality of devices of the private LAN according to a first set of probing requirements; and

collecting the descriptive information according to a second set of probing requirements, wherein said collecting the descriptive information according to the second set comprises collecting the descriptive information according to one or more probing requirements that are specific to an add-on application;

automatically reporting the collected information organized according to the first plurality of items of interest out of a firewall of the private LAN being crawled and over the Internet to a hosted on-demand search system;

receiving the reports from the private LAN being crawled at the hosted demand search system;

creating and synchronizing a private search database for the private LAN based on the received reports, wherein the private search database includes the descriptive information collected from across the disparate information sources of that private LAN organized by the first plurality of items of interest, and wherein the private search database is not organized by documents located on the private LAN;

receiving a search request to perform a private search of the private search database from the add-on application;

providing search results of the private search to the add-on application;

receiving at the hosted on-demand search system a first request from the private LAN to display the user interface with an application rendered component;

sending a second request to the add-on application to generate the application rendered component;

receiving from the add-on application a third request to perform one or more private searches for the descriptive information of the first plurality of items of interest in the private search database;

authorizing one or more private searches using a configuration database;

sending search results of the one or more private searches to the add-on application;

generating the application rendered component to represent the search results of the one or more private searches;

generating a page for the user interface that includes the application rendered component that represent the search results of the one or more private searches; and

sending the generated page, including the application rendered component, to the user interface to be displayed.

45. A method for hosted searching of private local area network (LAN) information, comprising:

automatically and repeatedly crawling a private LAN having a plurality of devices to collect, from across each of a plurality of disparate information sources stored in the plurality of devices of the private LAN, only those parts of the disparate information sources that include descriptive information of a first plurality of items of interest, wherein for at least one of the plurality items of interest, wherein for at least one of the plurality, items of interest, the parts are collected from multiple ones of the plurality of disparate information sources, wherein said automatically and repeatedly crawling comprises;

discovering the plurality of devices of the private LAN;

collecting the descriptive information from the disparate information sources stored on the plurality of devices of the private LAN according to a first set of probing requirements; and

collecting the descriptive information according to a second set of probing requirements, wherein said collecting the descriptive information according to the second set comprises collecting the descriptive information according to one or more probing requirements that are specific to an add-on application;

automatically reporting the collected information organized according to the first plurality of items of interest out of a firewall of the private LAN being crawled and over the Internet to a hosted on-demand search system;

receiving the reports from the private LAN being crawled at the hosted demand search system;

creating and synchronizing a private search database for the private LAN based on the received reports, wherein the private search database includes the descriptive information collected from across the disparate information sources of that private LAN organized by the first plurality of items of interest, and wherein the private search database is not organized by documents located on the private LAN;

receiving a search request to perform a private search of the private search database from the add-on application;

providing search results of the private search to the add-on application;

collecting from across the reports descriptive information of a second plurality of items of interest;

organizing the collected descriptive information of the second plurality of items of interest according to the second plurality of items of interest; and

integrating the collected descriptive information of the second plurality of items of interest organized according to the second plurality of items of interest into the corresponding private search database reorganized information into the corresponding private search databases;

wherein the descriptive information of the second plurality of items of interest include LAN-centric network topology information, wherein the method further comprises;

receiving at the hosted on-demand search system a first request from the private LAN to display a network topology map to be displayed on the user interface as part of the application rendered component;

sending a second request to the add-on application to generate the application rendered component that includes the network topology map;

receiving from the add-on application a third request to perform one or more private searches for LAN-centric network topology information in the private search database;

authorizing the one or more private searches using a configuration database;

sending search results of the one or more private searches to the add-on application;

generating the network topology map as part of the application rendered component based on the search results of the one or more private searches;

generating a page for the user interface that includes the application rendered component and the network topology map; and

sending the generated page, including the network topology map, to the user interface to be displayed.