Method and system for search engine indexing and searching using the index

• US 8,977,623 B2
• Filed: 05/05/2010
• Issued: 03/10/2015
• Est. Priority Date: 05/08/2009
• Status: Active Grant

• Current Assignee: Alibaba Group Holding Ltd.
• Original Assignee: Alibaba Group Holding Ltd.
• Inventors: Yang, Hanfei
• Primary Examiner(s): Perveen, Rehana
• Assistant Examiner(s): Hoffler, Raheem
• Application Number:
  US12/800,015
• Publication Number:
  US 20100287166A1
• Time in Patent Office: 1,770 Days
• Field of Search: None
• US Class Current: 707/741
• CPC Class Codes: G06F 16/2228 Indexing structures
• CPC Class Codes: G06F 16/285 Clustering or classification
• CPC Class Codes: G06F 16/316 Indexing structures
• CPC Class Codes: G06F 16/353 into predefined classes
• CPC Class Codes: G06F 16/90 Details of database functions independent of the retrieved data types
• CPC Class Codes: G06F 16/951 Indexing; Web crawling techniques

Abstract

Data indexing includes receiving data from a data source; classifying the data into one of a plurality of categories according to a predetermined data classification criteria; establishing a corresponding relationship between the data and an index associated with the data, the index having a preset maximum capacity; and recording the relationship between the data and the index. The index is one of a plurality of indices, and each of the plurality of indices is exclusively written by an index writing device.

Claims

1. A method for data indexing, comprising:

receiving data to be indexed from a data source;

classifying the data into one of a plurality of categories according to a predetermined data classification criteria;

selecting an index of a plurality of indices associated with the category to which the data was classified;

establishing a corresponding relationship between the data and the index, the index having a preset maximum capacity associated with an upper limit on an amount of data to be stored at the index; and

recording the relationship between the data and the index, wherein the index is configured to be exclusively written by a corresponding index writing device, wherein recording the relationship between the data and the index includes;

storing at least a portion of the data in a file associated with the index; and

storing an identifier associated with the index and an identifier associated with the corresponding index writing device in at least a portion of the file associated with the index.

2. The method of claim 1, further including:

receiving a search query;

determining an index in the plurality of indices that corresponds to the received search query; and

outputting stored data associated with the determined index.

3. The method of claim 1, further including determining a number of the plurality of indices based at least in part on a capacity of the category and the preset maximum capacity of the index.

4. The method of claim 1, wherein establishing the corresponding relationship between the data and the index includes

labeling the plurality of indices with consecutive positive integers.

5. The method of claim 1, wherein:

the index writing device is one of a plurality of index writing devices; and

recording the relationship between the data and the index includes;

labeling the plurality of index writing devices with consecutive positive integers;

dividing a serial number of the index by the number of the index writing devices to obtain a remainder value; and

associating the data and the corresponding index writing device that has a serial number that is the same as the remainder value.

6. The method of claim 1, wherein classifying the data into one of the plurality of categories is based at least in part on a property of the data.

7. The method of claim 1, further comprising storing the file and the data in a storage device.

8. The method of claim 1, wherein prior to recording the relationship between the data and the index, further comprising:

determining whether a total amount of data stored at the index exceeds the preset maximum capacity;

wherein in the event that the total amount of data stored at the index does not exceed the preset maximum capacity, recording the relationship between the data and the index,

wherein in the event that the total amount of data stored at the index does exceed the preset maximum capacity, adding a new index to the plurality of indices associated with the category and establishing a corresponding relationship between the data and the new index, and recording the relationship between the data and the new index.

9. A system for data indexing, comprising:

one or more processors configured to;

receive data to be indexed from a data source;

classify the data into one of a plurality of categories according to a predetermined data classification criteria;

select an index of a plurality of indices associated with the category to which the data was classified;

establish a corresponding relationship between the data and the index, the index having a preset maximum capacity associated with an upper limit on an amount of data to be stored at the index; and

record the relationship between the data and the index, wherein the index is configured to be exclusively written by a corresponding index writing device, wherein recording the relationship between the data and the index includes;

storing at least a portion of the data in a file associated with the index; and

storing an identifier associated with the index and an identifier associated with the corresponding index writing device in at least a portion of the file associated with the index;

and

one or more memories coupled to the one or more processors, configured to provide the processors with instructions.

10. The system of claim 9, wherein the one or more processors are further configured to:

receive a search query;

determine an index in the plurality of indices that corresponds to the received search query; and

output stored data associated with the determined index.

11. The system of claim 9, wherein the one or more processors are further configured to determine a number of the plurality of indices based at least in part on a capacity of the category and the preset maximum capacity of the index.

12. The system of claim 9, wherein establishing the corresponding relationship between the data and the index includes labeling the plurality of indices with consecutive positive integers.

13. The system of claim 9, wherein:

the index writing device is one of a plurality of index writing devices; and

recording the relationship between the data and the index includes;

labeling the plurality of index writing devices with consecutive positive integers;

dividing a serial number of the index by the number of the index writing devices to obtain a remainder value; and

associating the data and the corresponding index writing device that has a serial number that is the same as the remainder value.

14. The system of claim 9, wherein classifying the data into one of the plurality of categories is based at least in part on a property of the data.

15. The system of claim 9, wherein one or more processors are configured to store the file and the data in a storage device.

16. The system of claim 9, wherein prior to recording the relationship between the data and the index, the one or more processors are configured to:

determine whether a total amount of data stored at the index exceeds the preset maximum capacity;

wherein in the event that the total amount of data stored at the index does not exceed the preset maximum capacity, record the relationship between the data and the index,

wherein in the event that the total amount of data stored at the index does exceed the preset maximum capacity, add a new index to the plurality of indices associated with the category and establish a corresponding relationship between the data and the new index, and record the relationship between the data and the new index.