CRAWLABLE APPLICATIONS

• US 20080275844A1
• Filed: 05/01/2007
• Published: 11/06/2008
• Est. Priority Date: 05/01/2007
• Status: Active Grant

• Current Assignee: Oracle International Corporation (Oracle Corporation)
• Original Assignee: Oracle International Corporation (Oracle Corporation)
• Inventors: Lam, David, Hudson, Kevin, Wang, Anping, Buzsaki, George
• Granted Patent: US 7,752,207 B2
• Time in Patent Office: Days
• US Class Current: 707/3
• CPC Class Codes: G06F 16/245 Query processing
• CPC Class Codes: G06F 16/24547 Optimisations to support specific applications; Extensibility of optimisers

Abstract

Systems and methods in accordance with various embodiments of the present invention provide for a computer based method for crawling application data from an application data store. The applications data store has business objects of an application stored thereon. The method may include identifying a first request for application data received from a search engine as a seed universal resource locator (URL). A crawlable definition for the identified business object is accessed, the crawlable definition including a query selecting one or more attributes of the business object. Moreover, the method can include sending the query to the application data store and receiving query results in response thereto. Additionally, the method can include forming a crawlable document which includes the retrieved results of the business object.

Claims

1. A computer based method for crawling application data from an application data store having a plurality of business objects of an application stored thereon, the method comprising the steps of:

identifying a first request for application data received from a search engine as a seed universal resource locator (URL) sourcing a business object of the plurality of business objects of the application;

accessing a crawlable definition for the identified business object, the crawlable definition including a query selecting one or more attributes of the business object;

sending the query to the application data store and receiving query results in response thereto;

forming a crawlable document, including the retrieved results of the business object, to generate a data feed;

generating a control feed by dividing the data feed into transferrable batches and creating batch URLs for the batches; and

returning a feed document to the search engine, the feed document comprising the control feed.

2. The method of claim 1, wherein the seed URL sources one or more of the business objects of the application.

3. The method of claim 1, wherein communication with the application data store is performed over a virtual network over a defined web enabled messaging protocol.

4. The method of claim 1, wherein the feed document is further comprised of the data feed.

5. The method of claim 1, further comprising:

identifying a second request for application data received from a search engine as a re-entry URL;

returning a batch from the data feed where the batch is requested in the re-entry URL;

returning a related document where the related document is requested in the re-entry URL; and

returning a dependent document where the dependent document is requested in the re-entry URL.

6. The method of claim 1, wherein the transferrable batches are of a size capable of being transferred over a defined web enabled messaging protocol.

7. The method of claim 1, wherein the data feed is generated using the crawlable definition for the identified business object.

8. A system for crawling structured application data of an application, the system comprising:

a relational database operable to provide access to the application data stored in persistent storage;

a search engine operable to crawl a sourced business object of the application by sending a request for a seed universal resource locator (URL); and

an interface logically arranged between the database and the search engine, the interface operable to translate the request into a query by accessing a crawlable definition for the business object, the crawlable definition including the query for selecting one or more attributes of the business object, the interface being further operable to retrieve query results from the database and to translate the results into a form that is crawlable by the search engine by generating a data feed, the interface being operable to generating a control feed by dividing the data feed into transferrable batches and creating batch URLs for the batches, the interface being further operable to return a feed document comprising the control feed to the search engine.

9. The system of claim 8, wherein the feed document is further comprised of the data feed.

10. The system of claim 8, further comprising:

the interface operable to identify a second request for application data received from a search engine as a re-entry URL, the interface operable to return a batch from the data feed where the batch is requested in the re-entry URL, the interface operable to return a related searchable object where the related business object is requested in the re-entry URL, the interface operable to returning a dependent document where the dependent document is requested in the re-entry URL.

11. The system of claim 8, wherein the seed URL sources one or more of the business objects of the application.

12. The system of claim 8, wherein communication with the application data store is performed over a virtual network over a defined web enabled messaging protocol.

13. The system of claim 8, wherein the transferrable batches are of a size capable of being transferred over a defined web enabled messaging protocol.

14. The system of claim 8, wherein the data feed is generated using the crawlable definition for the identified business object.

15. A memory for storing data for access by an application program being executed on a data processing system, comprising:

a first set of metadata tags to translate a request by a search engine crawler for a business object of a plurality of business objects of an application, the first set of metadata tags comprising a query metadata tag to identify a query selecting one or more attributes of the business object stored in an application data store;

a second set of metadata tags to translate a query result to the search engine by specifying how the business object of the plurality of business objects is crawled by a search engine, the second set of metadata tags comprising a mapping metadata tag to identify an attribute of the one or more selected attributes to be mapped to a document structure persisted in the search engine; and

a third set of metadata tags to specify how the business object is displayed.

16. The memory of claim 15, wherein the second set of metadata tags comprises an indexing metadata tag to identify an attribute of the one or more selected attributes to be crawled by the search engine.

17. The memory of claim 15, wherein the second set of metadata tags comprises a display metadata tag to identify an attribute of the one or more selected attributes to be displayed in a URL.

18. The memory of claim 15, wherein the second set of metadata tags comprises a security metadata tag to identify an attribute of the one or more selected attributes as a security attribute.

19. A computer based method for enabling structured application data to be crawled via a user interface for the development of a crawlable business object definition, the method comprising:

providing a data source field for user input within a display window to set a data source definition for a business object of a plurality of business objects of an application, the data source definition including a query selecting one or more attributes of the business object to be exposed to a search engine; and

providing a plurality of user input fields corresponding to a first set of metadata tags to specify how each attribute of the one or more selected attributes of the business object is to be crawled by the search engine, the first set of metadata tags including a mapping metadata tag to identify an attribute of the one or more selected attributes to be mapped to a document structure persisted in the search engine.

20. The method of claim 19, further comprising:

providing a plurality of user input fields corresponding to a second set of metadata tags to specify how each attribute of the one or more selected attributes of the business object is displayed.

21. The method of claim 19, wherein the first set of metadata tags includes an indexing metadata tag to identify an attribute of the one or more selected attributes to be crawled by the search engine.

22. The method of claim 19, wherein the second set of metadata tags includes a display metadata tag to identify an attribute of the one or more selected attributes to be displayed in a URL.

23. The method of claim 19, wherein the first set of metadata tags includes a security metadata tag to identify an attribute of the one or more selected attributes as a security attribute.

24. A computer program product embedded in a computer readable medium, comprising:

program code for identifying a first request for application data received from a search engine as a seed universal resource locator (URL) sourcing a business object of the plurality of business objects of the application;

program code for accessing a crawlable definition for the identified business object, the crawlable definition including a query selecting one or more attributes of the business object;

program code for sending the query to the application data store and receiving query results in response thereto;

program code for forming a crawlable document, including the retrieved results of the business object, to generate a data feed;

program code for generating a control feed by dividing the data feed into transferrable batches and creating batch URLs for the batches; and

program code for returning a feed document to the search engine, the feed document comprising the control feed.

25. The computer program product according to claim 24, wherein the feed document is further comprised of the data feed.

26. The computer program product according to claim 24, further comprising:

program code for identifying a second request for application data received from a search engine as a re-entry URL;

program code for returning a batch from the data feed where the batch is requested in the re-entry URL;

program code for returning a related document where the related document is requested in the re-entry URL; and

program code for returning a dependent document where the dependent document is requested in the re-entry URL.

27. A method for crawling structured application data of an application by a search engine, the method comprising the steps of:

sending a first request for a sourced business object of a plurality of business objects of the application and receiving a control feed comprising one or more re-entry links in response thereto;

crawling the control feed to push the one or more re-entry links onto a job queue;

sending a second request for structured application data, the second request being for a re-entry link of the one or more re-entry links in the job queue;

receiving a crawlable batch from a data feed where the requested re-entry link is a batch link;

receiving a crawlable related document where the requested re-entry link is a related document link; and

receiving a crawlable dependent document where the requested re-entry link is a dependent document link.

28. The method of claim 27, further comprising:

indexing a content of the received crawlable batch according to an indexing rule set out in metadata of the received batch.

29. The method of claim 27, further comprising:

indexing a content of the received crawlable document according to an indexing rule set out in metadata of the received document.

30. The method of claim 27, further comprising:

indexing a content of the received crawlable dependent document according to an indexing rule set out in metadata of the received dependent document.