Method and apparatus for implementing a search index generator

• US 10,657,177 B1
• Filed: 12/31/2014
• Issued: 05/19/2020
• Est. Priority Date: 12/31/2014
• Status: Active Grant

• Current Assignee: Groupon Incorporated
• Original Assignee: Groupon Incorporated
• Inventors: Guo, Jiaqi, Subramanian, Sri, Gopalan, Ram
• Primary Examiner(s): Vy, Hung T
• Application Number:
  US14/588,196
• Time in Patent Office: 1,966 Days
• Field of Search: 707696, 707687
• CPC Class Codes: G06F 16/901 Indexing; Data structures therefor; Storage structures for retrieval from the web G06F16/951
• CPC Class Codes: G06F 16/90335 Query processing
• CPC Class Codes: G06F 16/951 Indexing; Web crawling techniques
• CPC Class Codes: G06Q 30/0207 Discounts or incentives, e.g. coupons or rebates

Abstract

The disclosure provides examples of methods, apparatuses, systems, and computer program products for implementing a search indexer. One example method includes receiving, by index workflow circuitry, a search index update indicator, causing, by the index workflow circuitry and in response to receiving the search index update indicator, index generation circuitry to retrieve data sets from a plurality of data sources, calculating a checksum for each of the data sets, comparing the checksum for each of the data sets to a set of search index metadata, in response to determining that at least one checksum for at least one of the data sets does not match a metadata checksum from the set of search index metadata, generating a set of search index data from the data set corresponding to the at least one checksum, generating a search index using at least the set of search index data, and transmitting the search index to a search service for use in responding to at least one search query.

Claims

1. A method for generating a search index in a system where relationships between searchable items are dynamically changing and relationships between searchable items and search query criteria are dynamically changing, the method comprising:

receiving, by index workflow circuitry comprising a processor, a search index update indicator, the search index update indicator generated in response to a change in one or more relationships between searchable items and search query criteria based at least in part on interactions with promotion content by consumer devices;

causing, by the index workflow circuitry comprising the processor, and in response to receiving the search index update indicator, index generation circuitry to retrieve data sets from a plurality of data sources, wherein the plurality of data sources includes relevance data identifying relationships between interactions with content and characteristics of the content;

calculating, by the index workflow circuitry comprising the processor, a checksum for each of the data sets;

comparing, by the index workflow circuitry comprising the processor, the checksum for each of the data sets to a set of search index metadata;

in response to determining that at least one checksum for at least one of the data sets does not match a metadata checksum from the set of search index metadata, generating, by the index workflow circuitry comprising the processor, a set of search index data from the data set corresponding to the at least one checksum;

generating, by the index workflow circuitry comprising the processor, a search index using at least the set of search index data; and

transmitting, by the index workflow circuitry comprising the processor, the search index to a search service for use in responding to at least one search query.

2. The method of claim 1, further comprising:

receiving, by the index workflow circuitry comprising the processor, a set of search results from the search service;

determining, by the index workflow circuitry comprising the processor, a set of content corresponding to the set of search results by ranking the set of search results according to one or more ranking algorithms;

providing, by the index workflow circuitry comprising the processor, the set of content to at least one consumer.

3. The method of claim 1, wherein at least two of the data sets are located at different data centers.

4. The method of claim 1, wherein the search index comprises a set of JavaScript Object Notation pair values.

5. The method of claim 4, wherein the pair values comprise a search term and a content identifier.

6. The method of claim 1, wherein the search index update indicator is a new data set registering with the index workflow circuitry.

7. The method of claim 1, wherein the search index update indicator is a periodic indicator.

8. The method of claim 1, further comprising excluding, by the index workflow circuitry comprising the processor, a data set in response to determining that the at least one checksum matches at least one metadata checksum.

9. The method of claim 1, further comprising splitting the at least one data set into multiple shards.

10. The method of claim 9, wherein each of the multiple shards is processed concurrently.

11. The method of claim 10, wherein each of the multiple shards is processed as a separate MapReduce job.

12. The method of claim 1, wherein generating the search index comprises at least one MapReduce job.

13. The method of claim 1, wherein the plurality of data sources further includes one or more of relevance data identifying relationships between particular consumers and particular promotions, data related to past promotion purchase for consumers, data associating locations with particular promotions, data associating particularly successful marketing campaigns with particular promotions, and data identifying relationships between promotions and particular types of categories.

14. An apparatus for generating a search index in a system where relationships between searchable items are dynamically changing and relationships between searchable items and search query criteria are dynamically changing, the apparatus comprising:

means for receiving a search index update indicator, the search index update indicator generated in response to a change in one or more relationships between searchable items and search query criteria based at least in part on interactions with promotion content by consumer devices;

means for retrieving, in response to receiving the search index update indicator, data sets from a plurality of data sources, wherein the plurality of data sources includes relevance data identifying relationships between interactions with content and characteristics of the content;

means for calculating a checksum for each of the data sets;

means for comparing the checksum for each of the data sets to a set of search index metadata;

means for generating, in response to determining that at least one checksum for at least one of the data sets does not match a metadata checksum from the set of search index metadata, a set of search index data from the data set corresponding to the at least one checksum;

means for generating a search index using at least the set of search index data; and

means for transmitting the search index to a search service for use in responding to at least one search query.

15. An apparatus for generating a search index in a system where relationships between searchable items are dynamically changing and relationships between searchable items and search query criteria are dynamically changing, the apparatus comprising:

index workflow circuitry and index generation circuitry, the index workflow circuitry and the index generation circuitry employing one or more processors;

wherein the index workflow circuitry is configured to, using one or more processors;

receive a search index update indicator, the search index update indicator generated in response to a change in one or more relationships between searchable items and search query criteria based at least in part on interactions with promotion content by consumer devices;

cause, in response to receiving the search index update indicator, the index generation circuitry to retrieve data sets from a plurality of data sources, wherein the plurality of data sources includes relevance data identifying relationships between interactions with content and characteristics of the content;

calculate a checksum for each of the data sets;

compare the checksum for each of the data sets to a set of search index metadata; and

transmit a search index to a search service for use in responding to at least one search query; and

wherein the index generation circuitry is configured to, using one or more processors;

generate, in response to determining that at least one checksum for at least one of the data sets does not match a metadata checksum from the set of search index metadata, a set of search index data from the data set corresponding to the at least one checksum; and

generate the search index using at least the set of search index data.

16. The apparatus of claim 15, further comprising:

communications circuitry employing one or more processors, wherein the communications circuitry is configured to, with the one or more processors;

receive a set of search results from the search service; and

provide the set of content to at least one consumer; and

a processor configured to;

determine a set of content corresponding to the set of search results by ranking the set of search results according to one or more ranking algorithms.

17. The apparatus of claim 15, wherein at least two of the data sets are located at different data centers.

18. The apparatus of claim 15, wherein the search index comprises a set of JavaScript Object Notation pair values.

19. The apparatus of claim 15, wherein the pair values comprise a search term and a content identifier.

20. The apparatus of claim 15, wherein the search index update indicator is one of a new data set registering with the index workflow circuitry or a periodic indicator.