Automatically obtaining real-time, geographically-relevant product information from heterogeneus sources

• US 9,977,790 B2
• Filed: 02/06/2012
• Issued: 05/22/2018
• Est. Priority Date: 02/04/2011
• Status: Active Grant

• Current Assignee: eBay Inc.
• Original Assignee: eBay Inc.
• Inventors: Abraham, Jack Phillip, Adelson, Aaron, Barto, Matthew, Dziuba, Theodore James, Evans, John, Newey, Neville, Van Winkle, Justin
• Primary Examiner(s): Bibbee, Jared M
• Application Number:
  US13/366,962
• Publication Number:
  US 20120203760A1
• Time in Patent Office: 2,297 Days
• Field of Search: 707710
• CPC Class Codes: G06F 16/29 Geographical information databases
• CPC Class Codes: G06F 16/951 Indexing; Web crawling techniques

Abstract

Techniques for obtaining geographically-relevant product inventory information, in real-time, from heterogeneous data sources are described. Product inventory information, including the volume of available products in specific geographical locations, is obtained from at least three different sources. First, one or more data feeds may be received. Second, a data obtaining module uses one or more APIs to obtain product inventory information from one or more third-party inventory management systems. Finally, a structured data mining module uses a web crawler, at the direction of a crawler configuration, to systematically obtain product inventory information from various third-party websites. Accordingly, a user'"'"'s search query is processed to provide geographically relevant product inventory information in near real time.

Claims

1. A method for populating a database with product inventory information, the method comprising:

with one or more processors, executing a first set of instructions representing an instance of a crawler application, the instance of the crawler application to perform a set of operations specified in a crawler configuration file that has been generated with a web-based crawler configuration application, the performance of the set of operations resulting in the retrieval of product inventory information for one or more products hosted at one or more web servers;

with at least one of the one or more processors, executing a second set of instructions representing an instance of a real-time product availability lookup (RTPAL) application, the RTPAL application to use one or more application programming interfaces (APIs) to request and receive product inventory information from one or more third-party network-connected inventory management systems;

subsequent to receiving product inventory information as a result of executing either of the crawler application and the RTPAL application, determining that received product inventory information for a particular product does not specify a unique product identifier;

performing a matching operation by comparing various elements of information concerning the particular product with corresponding information from one or more known products to determine a product with which the received product inventory information for the particular product not specifying the unique product identifier best matches;

storing the received product inventory information for the particular product not specifying the unique product identifier in the database;

receiving a search query from a user, the search query including information identifying a desired geographical area;

processing the search query to provide product inventory information for a particular product satisfying the search query, the product inventory information for the particular product satisfying the search query specifying one or more merchant stores in a geographical location satisfying the desired geographical area identified in the query, and indicating a quantity of the particular product satisfying the search query available at each of the one or more merchant stores.

2. The method of claim 1, wherein the web-based crawler configuration application enables the user to specify one of a suite of web crawlers for use with a particular crawler configuration file generated by the web crawler configuration application.

3. The method of claim 1, wherein the web-based crawler configuration application enables the user to invoke a code editing application to specify customized code for obtaining a particular element of product inventory information, the customized code for inclusion in a crawler configuration file for use with a particular crawler.

4. The method of claim 1, further comprising:

storing product inventory information for a particular product received via the crawler application or the RTPAL application in a data cache with each cache entry having a cache key based on a zipcode for a location at which the particular product received via the crawler application or the RTPAL application is available, a product identifier, and a product variation identifier.

5. The method of claim 1, further comprising:

assigning to each product identified in the received product inventory information as a result of executing either of the crawler application and the RTPAL application one or more category identifiers for a particular category in a hierarchical category.

6. A machine-readable storage medium storing instructions thereon, which, when executed by a processor of a server, will cause the server to perform a set of operations comprising:

executing a first set of instructions representing an instance of a crawler application, the instance of the crawler application to perform a set of operations specified in a crawler configuration file that has been generated with a web-based crawler configuration application, the performance of the set of operations resulting in the retrieval of product inventory information for one or more products hosted at one or more web servers;

executing a second set of instructions representing an instance of a real-time product availability lookup (RTPAL) application, the RTPAL application to use one or more application programming interfaces (APIs) to request and receive product inventory information from one or more third-party network-connected inventory management systems;

subsequent to receiving product inventory information as a result of executing either of the crawler application and the RTPAL application, determining that received product inventory information for a particular product does not specify a unique product identifier;

performing a matching operation by comparing various elements of information concerning the particular product with corresponding information from one or more known products to determine a product with which the received product inventory information for the particular product not specifying the unique product identifier best matches;

storing the product inventory information for the particular product not specifying the unique product identifier in a database;

storing product inventory information for a particular product received via the crawler application or the RTPAL application in a data cache with each cache entry having a cache key based on a zipcode for a location at which the product is available, a product identifier, and a product variation identifier.

7. The machine-readable medium of claim 6, wherein the web-based crawler configuration application enables a user to specify one of a suite of web crawlers for use with a particular crawler configuration file generated by the web crawler configuration application.

8. The machine-readable medium of claim 6, wherein the web-based crawler configuration application enables a user to invoke a code editing application to specify customized code for obtaining a particular element of product inventory information, the customized code for inclusion in a crawler configuration file for use with a particular crawler.

9. The machine-readable medium of claim 6, wherein the operations further comprise:

assigning to each product identified in the received product inventory information as a result of executing either of the crawler application and the RTPAL application one or more category identifiers for a particular category in a hierarchical category.

10. The machine-readable medium of claim 6, wherein the operations further comprise:

receiving a search query from a user, the search query including information identifying a desired geographical area;

processing the user'"'"'s search query to provide product inventory information for a particular product satisfying the search query, the product inventory information specifying one or more merchant stores in a geographical location satisfying the desired geographical area identified in the query, and indicating a quantity of the particular product satisfying the search query available at each of the one or more merchant stores.

11. A product search engine comprising:

one or more processors for executing one or more sets of instructions stored in memory;

a first set of executable instructions stored in the memory and representing an instance of a crawler application, which, when executed, is to perform a set of operations specified in a crawler configuration file that has been generated with a web-based crawler configuration application, the performance of the set of operations resulting in the retrieval of product inventory information for one or more products hosted at one or more web servers;

a second set of executable instructions stored in the memory and representing an instance of a real-time product availability lookup (RTPAL) application, the RTPAL application, when executed, to use one or more application programming interfaces (APIs) to request and receive product inventory information from one or more third-party network-connected inventory management systems;

a third set of executable instructions representing a product offering matching module, which, when executed, is to determine that received product inventory information for a particular product does not specify a unique product identifier and then perform a matching operation by comparing various elements of information concerning the particular product with corresponding information from one or more known products to determine a product with which the received product inventory information for the particular product not specifying the unique product identifier best matches;

a database module to store the product inventory information for the particular product not specifying the unique product identifier in a database; and

a cache module for storing the product inventory information for a particular product received via the crawler application or the RTPAL application in a data cache with each cache entry having a cache key based on a zipcode for a location at which the product is available, a product identifier, and a product variation identifier.

12. The method of claim 11, wherein the web-based crawler configuration application enables a user to specify one of a suite of web crawlers for use with a particular crawler configuration file generated by the web crawler configuration application.

13. The method of claim 11, wherein the web-based crawler configuration application enables a user to invoke a code editing application to specify customized code for obtaining a particular element of product inventory information, the customized code for inclusion in a crawler configuration file for use with a particular crawler.

14. The method of claim 11, further comprising a categorization module to assign to each product identified in the received product inventory information as a result of executing either of the crawler application and the RTPAL application one or more category identifiers for a particular category in a hierarchical category.

15. The method of claim 11, further comprising a search module to:

receive a search query from a user, the search query including information identifying a desired geographical area;

process the user'"'"'s search query to provide product inventory information for a particular product satisfying the search query, the product inventory information for the particular product satisfying the search query specifying one or more merchant stores in a geographical location satisfying the desired geographical area identified in the query, and indicating a quantity of the particular product available at each of the one or more merchant stores.