Systems and methods to build and utilize a search infrastructure

• US 9,607,049 B2
• Filed: 12/28/2012
• Issued: 03/28/2017
• Est. Priority Date: 07/25/2012
• Status: Active Grant

• Current Assignee: eBay Inc.
• Original Assignee: eBay Inc.
• Inventors: He, Gang, Agarwal, Swati, Henderson, Richard D, Libenzi, Davide, Nallapaneni, Jagadish, Sankaranthi, Pradeep, Whyte, Nicholas, Pan, Thomas, Hoffacker, Carson, Basu, Amit, Sun, Mel, Melnick, Andrew John
• Primary Examiner(s): Morrison, Jay
• Assistant Examiner(s): Hoang, Ken
• Application Number:
  US13/730,536
• Publication Number:
  US 20140032566A1
• Time in Patent Office: 1,551 Days
• Field of Search: 707/E17.005, 707/E17.01, 707/E17.014, 707/639, 707/769, 707/E17.007, 707/E17.032, 707/634, 707/741, 707/999.003, 707/E17.001, 707/E17.002, 707/E17.009, 707/E17.044, 707/E17.108, 707/610, 707/611, 707/624, 707/758, 707/770, 707/802, 707/999.101, 707/999.102, 707/E17.012, 707/E17.033, 707/E17.052, 707/E17.135, 707/E17.136, 707/609, 707/636, 707/638, 707/650, 707/655, 707/662, 707/672, 707/673, 707/674, 707/685, 707/688, 707/690, 707/695, 707/709, 707/719, 707/747, 707/776, 707/797, 707/812, 707/823, 707/827, 707/999.001, 707/999.004, 707/999.005, 707/999.1, 707/999.107, 707/999.202, 707/999.203, 707/999.205, 711/E12.103, 711/E12.001, 711/162, 711/112, 711/118, 711/128, 711/156
• US Class Current: 1/1
• CPC Class Codes: G06F 16/2228 Indexing structures
• CPC Class Codes: G06F 16/245 Query processing
• CPC Class Codes: G06F 16/24578 using ranking
• CPC Class Codes: G06F 16/316 Indexing structures
• CPC Class Codes: G06F 16/319 Inverted lists
• CPC Class Codes: G06F 16/337 Profile generation, learning or modification
• CPC Class Codes: G06F 16/93 Document management systems

Abstract

Methods and systems to build and utilize a search infrastructure are described. The system generates index information components in real-time based on a database that is time-stamped. The system updates index information at a plurality of query node servers based on the index information components. A query engine receives a search query from a client machine and identifies search results based on the query and the index information. The system communicates the search results, over the network, to the client machine.

Claims

1. A system comprising:

a map-reduce job module to generate index information components in real-time based on a database that is time-stamped, the index information components comprise a full-index and a plurality of mini-indexes, the full-index comprises a snapshot of contents of the database at a first time, the plurality of mini-indexes respectively corresponding to a plurality of snapshots of updates to the database that correspond to a plurality of equal length time periods comprised of a start-time and an end-time;

a query engine to update the index information at a plurality of query node servers based on the index information components, the query engine to further receive a search query, over a network, from a client machine and identify search results based on the search query and the index information components; and

a front-end server to communicate the search results, over the network, to the client machine, wherein the plurality of query node servers includes a first query node server, wherein the plurality of mini-indexes includes a first mini-index and a second mini-index, wherein the query engine updates the index information components at the first query node server with the full-index and with the plurality of mini-indexes, updates the first query node server based on the first mini-index, and blocks an update of the index information components in the first query node server responsive to an identification of the second mini-index as arriving out of sequence.

2. The system of claim 1, wherein the full-index is associated with full-index bill of material information and the first mini-index is associated with a first mini-index bill of material information, wherein the full-index bill of material information includes a full-index identifier and the first mini-index bill of material information includes a full-index identifier.

3. The system of claim 2, wherein the query engine compares the full-index identifier in the full-index bill of material information with the full-index identifier in the mini-index bill of material information.

4. The system of claim 1, wherein the query engine blocks an update of the index information components in the first query node server responsive to receipt of the second mini-index and identification of the first mini-index as not stored in mini-index storage information.

5. The system of claim 1, wherein the query engine blocks an update of the index information components in the first query node server responsive to an identification that a received mini-index is not compatible with the full-index that was utilized to update the index information.

6. A method comprising:

generating index information components in real-time based on a database that is time-stamped, the index information components comprising a full-index and a plurality of mini-indexes, the full-index comprises a snapshot of contents of the database at a first time, the plurality of mini-indexes respectively corresponding to a plurality of snapshots of updates to the database that correspond to a plurality of equal length time periods comprised of a start-time and an end-time;

updating index information at a plurality of query node servers based on the index information components;

receiving a search query, over a network, from a client machine and identifying search results based on the search query and the index information components; and

communicating the search results, over the network, to the client machine, wherein the plurality of query node servers includes a first query node server, wherein the plurality of mini-indexes includes a first mini-index and a second mini-index, wherein the query engine updates the index information components at the first query node server with the full-index and with the plurality of mini-indexes, updates the first query node server based on the first mini-index, and blocks an update of the index information components in the first query node server responsive to an identification of the second mini-index as arriving out of sequence.

7. The method of claim 6, wherein the full-index is associated with full-index bill of material information and the first mini-index is associated with a first mini-index bill of material information, wherein the full-index bill of material information includes a full-index identifier and the first mini-index bill of material information includes a full-index identifier.

8. The method of claim 7, wherein the updating the index information comprises comparing the full-index identifier in the full-index bill of material information with the full-index identifier in the mini-index bill of material information.

9. The method of claim 6, wherein the updating the index information comprises blocking the first query node server from updating the index information components in the first query node server responsive to receiving the second mini-index and identifying the first mini-index as not being stored in mini-index storage information.

10. The method of claim 6, wherein the updating the index information components comprises the first query node server from updating the index information in the first query node server responsive to identifying that a received mini-index is not compatible with the full-index that was utilized to update the index information.

11. A non-transitory machine-readable medium comprising instructions, which when executed on a processor, cause the processor to perform a method comprising:

generating index information components in real-time based on a database that is time-stamped, the index information components comprising a full-index and a plurality of mini-indexes, the full-index comprises a snapshot of contents of the database at a first time, the plurality of mini-indexes respectively corresponding to a plurality of snapshots of updates to the database that correspond to a plurality of equal length time periods comprised of a start-time and an end-time;

updating index information at a plurality of query node servers based on the index information components;

receiving a search query, over a network, from a client machine and identifying search results based on the search query and the index information components; and

communicating the search results, over the network, to the client machine, wherein the plurality of query node servers includes a first query node server, wherein the plurality of mini-indexes includes a first mini-index and a second mini-index, wherein the query engine updates the index information components at the first query node server with the full-index and with the plurality of mini-indexes u s dates the first query node server based on the first mini-index and blocks an update of the index information components in the first query node server responsive to an identification of the second mini-index as arriving out of sequence.

12. A system comprising:

a means to generate index information components in real-time based on a database that is time-stamped, the index information components comprise a full-index and a plurality of mini-indexes, the full-index comprises a snapshot of contents of the database at a first time, the plurality of mini-indexes respectively corresponding to a plurality of snapshots of updates to the database that correspond to a plurality of equal length time periods comprised of a start-time and an end-time;

a query engine to update index information at a plurality of query node servers based on the index information components, the query engine to further receive a search query, over a network, from a client machine and identify search results based on the search query and the index information components; and

a front-end server to communicate the search results, over the network, to the client machine, wherein the plurality of query node servers includes a first query node server, wherein the plurality of mini-indexes includes a first mini-index and a second mini-index, wherein the query engine updates the index information components at the first query node server with the full-index and with the plurality of mini-indexes, updates the first query node server based on the first mini-index, and blocks an update of the index information components in the first query node server responsive to an identification of the second mini-index as arriving out of sequence.