Techniques for determining a semantic distance between subjects

• US 10,621,219 B2
• Filed: 02/10/2017
• Issued: 04/14/2020
• Est. Priority Date: 02/10/2017
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: English, Jennifer Ann, Kossarian, Malous Melissa, McManis, Jr., Charles E., Smith, Douglas A.
• Primary Examiner(s): Beausoliel, Jr., Robert W
• Assistant Examiner(s): Hoang, Hau
• Application Number:
  US15/430,185
• Publication Number:
  US 20180232437A1
• Time in Patent Office: 1,159 Days
• CPC Class Codes: G06F 16/313 Selection or weighting of terms for indexing
• CPC Class Codes: G06F 16/3344 using natural language analysis
• CPC Class Codes: G06F 16/3347 using vector based model
• CPC Class Codes: G06F 40/30 Semantic analysis
• CPC Class Codes: G06F 40/44 Statistical methods, e.g. probability models
• CPC Class Codes: G06N 20/00 Machine learning
• CPC Class Codes: G06N 7/01 Probabilistic graphical models, e.g. probabilistic networks

Abstract

A technique for calculating a semantic distance between subjects includes performing a mathematical operation between each of one or more first topic vectors and each of one or more second topic vectors to generate respective strength values. The first topic vectors are associated with respective first topics of a first subject, the second topic vectors are associated with respective second topics of a second subject, and the respective strength values are indicative of a relative closeness between associated ones of the first and second topics. Relevant ones of the respective strength values are summed to provide an overall strength value between the first subject and the second subject. A semantic distance between the first subject and the second subject is determined based on the overall strength value.

Claims

1. A computer program product configured to determine a semantic distance between subjects, the computer program product comprising:

a computer-readable storage device; and

computer-readable program code embodied on the computer-readable storage device, wherein the computer-readable program code, when executed by a data processing system, causes the data processing system to;

perform a mathematical operation between each of one or more first topic vectors and each of one or more second topic vectors to generate respective strength values, wherein the first topic vectors are associated with respective first topics of a first subject, the second topic vectors are associated with respective second topics of a second subject, and the respective strength values are indicative of a relative closeness between associated ones of the first and second topics;

sum relevant ones of the respective strength values to provide an overall strength value between the first subject and the second subject;

determine a semantic distance between the first subject and the second subject based on the overall strength value; and

utilize both the first and second subjects in a search for information related to the first subject in response to the semantic distance being within a threshold distance value to improve the operation of the data processing system in answering a question about the first subject.

2. The computer program product of claim 1, wherein the computer-readable program code, when executed by the data processing system, further causes the data processing system to:

generate the first topic vectors and the second topic vectors based on a statistical model analysis.

3. The computer program product of claim 2, wherein the mathematical operation is a dot product operation.

4. The computer program product of claim 2, wherein the statistical model analysis is a latent Dirichlet allocation (LDA) analysis.

5. The computer program product of claim 1, wherein a number of the first topics for the first subject is determined by taking the square root of a number of documents associated with the first subject divided by two.

6. The computer program product of claim 5, wherein the documents are located using a Web search.

7. The computer program product of claim 1, wherein the semantic distance between the first subject and the second subject is determined by taking an inverse of the overall strength value.

8. The computer program product of claim 1, wherein each of the first and second topic vectors have an associated word and an associated strength that is normalized to one.

9. A data processing system, comprising:

a cache memory; and

a processor coupled to the cache memory, wherein the processor is configured to;

perform a mathematical operation between each of one or more first topic vectors and each of one or more second topic vectors to generate respective strength values, wherein the first topic vectors are associated with respective first topics of a first subject, the second topic vectors are associated with respective second topics of a second subject, and the respective strength values are indicative of a relative closeness between associated ones of the first and second topics;

sum relevant ones of the respective strength values to provide an overall strength value between the first subject and the second subject;

determine a semantic distance between the first subject and the second subject based on the overall strength value; and

utilize both the first and second subjects in a search for information related to the first subject in response to the semantic distance being within a threshold distance value to improve the operation of the data processing system in answering a question about the first subject.

10. The data processing system of claim 9, wherein the processor is further configured to:

generate the first topic vectors and the second topic vectors based on a statistical model analysis.

11. The data processing system of claim 10, wherein the mathematical operation is a dot product operation, the statistical model analysis is a latent Dirichlet allocation (LDA) analysis, and a number of the first topics for the first subject is determined by taking the square root of a number of documents associated with the first subject divided by two.

12. The data processing system of claim 9, wherein the semantic distance between the first subject and the second subject is determined by taking an inverse of the overall strength value.