FULL TEXT INDEXING IN A DATABASE SYSTEM

• US 20170329839A1
• Filed: 05/10/2016
• Published: 11/16/2017
• Est. Priority Date: 05/10/2016
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Fan, Si Bin, Jiang, Peng Hui, Sun, Lin, Sun, Yan, Xiang, Li, Xu, Yan, Yin, Wen
• Granted Patent: US 10,210,241 B2
• Time in Patent Office: Days
• CPC Class Codes: G06F 16/313 Selection or weighting of terms for indexing
• CPC Class Codes: G06F 16/338 Presentation of query results
• CPC Class Codes: G06F 16/93 Document management systems

Abstract

A method for indexing with redundant information. The method may identify unknown code points for a document in response to an indexing request for the document. The method may further convert the identified unknown code points into a plurality of converted code points. Each set of converted code points of the plurality uses a different codepage. The method may further identify sets of same code points and sets of redundant code points from the plurality of converted code points. The method may build an index based on the sets of same code points and the sets of redundant code points.

Claims

1. A processor-implemented method for indexing with redundant information, the method comprising:

identifying, by a processor, a plurality of unknown code points for a document in response to an indexing request for the document;

converting the identified plurality of unknown code points into a plurality of converted code points, wherein each of the plurality of converted code points uses a different codepage;

identifying sets of same code points and sets of redundant code points from the plurality of converted code points; and

building an index based on the identified sets of same code points and the identified sets of redundant code points.

2. The method of claim 1, wherein the converting the identified plurality of unknown code points into the plurality of converted code points further comprises:

converting the identified plurality of unknown code points into a first set of converted code points with a first codepage; and

converting the identified plurality of unknown code points into a second set of converted code points with a second codepage.

3. The method of claim 2, further comprising:

retaining the sets of same code points and the sets of redundant code points.

4. The method of claim 3, wherein the retaining the sets of same code points further comprises:

retaining a first set of same code points from the first set of converted code points; and

retaining a second set of same code points from the second set of converted code points.

5. The method of claim 2, further comprising:

assigning a weight to the first set of converted code points and the second set of converted code points.

6. The method of claim 5, wherein, in the built index, a redundant index is constructed from the first set of redundant code points and the second set of redundant code points, and is assigned with a same weight as the first set of converted code points and the second set of converted code points.

7. The method of claim 6, further comprising:

presenting a plurality of documents as a search result list in response to a search request, wherein a position of a search result document in the search result list is based on redundant indexes used by the search result document.

8. A computer system for indexing with redundant information, the computer system comprising:

one or more processors, one or more computer-readable memories, one or more computer-readable tangible storage medium, and program instructions stored on at least one of the one or more tangible storage medium for execution by at least one of the one or more processors via at least one of the one or more memories, wherein the computer system is capable of performing a method comprising;

identifying, by a processor, plurality of unknown code points for a document in response to an indexing request for the document;

converting the identified plurality of unknown code points into a plurality of converted code points, wherein each of the plurality of converted code points uses a different codepage;

identifying sets of same code points and sets of redundant code points from the plurality of converted code points; and

building an index based on the identified sets of same code points and the identified sets of redundant code points.

9. The computer system of claim 8, wherein the converting the identified plurality of unknown code points into the plurality of converted code points further comprises:

converting the identified plurality of unknown code points into a first set of converted code points with a first codepage; and

converting the identified plurality of unknown code points into a second set of converted code points with a second codepage.

10. The computer system of claim 9, further comprising:

retaining the sets of same code points and the sets of redundant code points.

11. The computer system of claim 10, wherein the retaining the sets of same code points further comprises:

retaining a first set of same code points from the first set of converted code points; and

retaining a second set of same code points from the second set of converted code points.

12. The computer system of claim 9, further comprising:

assigning a weight to the first set of converted code points and the second set of converted code points.

13. The computer system of claim 12, wherein, in the built index, a redundant index is constructed from the first set of redundant code points and the second set of redundant code points, and is assigned with a same weight as the first set of converted code points and the second set of converted code points.

14. The computer system of claim 13, further comprising:

presenting a plurality of documents as a search result list in response to a search request, wherein a position of a search result document in the search result list is based on redundant indexes used by the search result document.

15. A computer program product for indexing with redundant information, the computer program product comprising:

one or more computer-readable storage medium and program instructions stored on at least one of the one or more tangible storage medium, the program instructions executable by a processor, the program instructions comprising;

program instructions to identify, by a processor, plurality of unknown code points for a document in response to an indexing request for the document;

program instructions to convert the identified plurality of unknown code points into a plurality of converted code points, wherein each of the plurality of converted code points uses a different codepage;

program instructions to identify sets of same code points and sets of redundant code points from the plurality of converted code points; and

program instructions to build an index based on the identified sets of same code points and the identified sets of redundant code points.

16. The computer program product of claim 15, wherein the converting the identified plurality of unknown code points into the plurality of converted code points further comprises:

program instructions to convert the identified plurality of unknown code points into a first set of converted code points with a first codepage; and

program instructions to convert the identified plurality of unknown code points into a second set of converted code points with a second codepage.

17. The computer program product of claim 16, further comprising:

program instructions to retain the sets of same code points and the sets of redundant code points.

18. The computer program product of claim 17, wherein the retaining the sets of same code points further comprises:

program instructions to retain a first set of same code points from the first set of converted code points; and

program instructions to retain a second set of same code points from the second set of converted code points.

19. The computer program product of claim 16, further comprising:

program instructions to assign a weight to the first set of converted code points and the second set of converted code points.

20. The computer program product of claim 19, wherein, in the built index, a redundant index is constructed from the first set of redundant code points and the second set of redundant code points, and is assigned with a same weight as the first set of converted code points and the second set of converted code points, further comprising:

program instructions to present a plurality of documents as a search result list in response to a search request, wherein a position of a search result document in the search result list is based on redundant indexes used by the search result document.