Method, device, and computer storage media for adding hyperlink to text

• US 9,483,447 B2
• Filed: 02/08/2013
• Issued: 11/01/2016
• Est. Priority Date: 03/29/2012
• Status: Active Grant

• Current Assignee: Tencent Technology Company Limited (Tencent Holdings Limited)
• Original Assignee: Tencent Technology Company Limited (Tencent Holdings Limited)
• Inventors: He, Xiang, Bian, Qi, Jiao, Feng
• Primary Examiner(s): Bloomquist, Keith
• Application Number:
  US14/349,016
• Publication Number:
  US 20140250356A1
• Time in Patent Office: 1,362 Days
• US Class Current: 1/1
• CPC Class Codes: G06F 16/9558 Details of hyperlinks; Management of linked annotations
• CPC Class Codes: G06F 16/958 Organisation or management of web site content, e.g. publishing, maintaining pages or automatic linking
• CPC Class Codes: G06F 40/134 Hyperlinking
• CPC Class Codes: G06F 40/284 Lexical analysis, e.g. tokenisation or collocates

Abstract

Methods and devices for adding hyperlink to text are disclosed: generating hyperlink word list and characteristic word list in advance, determining co-occurrence frequency with each hyperlink word; to each text X which to be added the hyperlink, word segmentation processing them respectively, extracting the hyperlink word occurred in the hyperlink word list and the characteristic word occurred in the characteristic word list from results of word segmentation, determining weights of each extracted hyperlink word and extracted characteristic word, getting final weights of each extracted hypertext link word according to the co-occurrence frequency of each extracted characteristic word and each extracted hyperlink word and the weights; descendingly sorting each extracted hyperlink word according to the final weights, adding hyperlink to first k hyperlink words, and K is positive integer. Applying the solution, it can improve the relativity of the added hyperlink and the text, and it is easy to implement.

Claims

1. A method for adding hyperlinks to hyperlink words in a text, comprising:

creating a hyperlink word list in advance, the hyperlink word list comprising a plurality of hyperlink words;

collecting a variety of texts, and generating a characteristic word list by implementing word segmentation processing for each of the texts, the characteristic word list comprising a plurality of characteristic words;

respectively determining an IDF (inverse document frequency) value for each characteristic word after generating a characteristic word list by implementing word segmentation processing for each of the texts, wherein the IDF value is calculated by following processes;

obtaining a quotient by a quantity of the variety of texts collected divided by a quantity of texts appearing the characteristic word, and calculating a logarithm of the quotient;

for each of the characteristic words, computing a co-occurrence frequency between each of the characteristic words and each of the hyperlink words;

considering each text to be added a hyperlink as a text X, and processing the text X by following steps;

carrying out the word segmentation processing to the text X, and obtaining a segmentation result;

extracting the hyperlink words occurred in the hyperlink word list and the characteristic words occurred in the characteristic word list from the segmentation result;

computing a weight of each of the hyperlink words that are occurred in the hyperlink word list, and computing a weight of each of the characteristic words that are occurred in the characteristic word list, which comprises;

for each hyperlink word H, calculating the weight WH of the hyperlink word H;

WH=TFH*IDFH;

wherein, TFH represents TF (term frequency) value of the hyperlink word H and TFH refers to the quantity of the hyperlink word H appearing in the text X, and IDFH represents IDF value of the hyperlink word H;

for each characteristic word F, calculating the weight WH of the characteristic word F;

WF=TFF*IDFF;

wherein, TFF represents TF value of the characteristic word F, and IDFF represents IDF value of the characteristic word F;

determining a final weight of each of the hyperlink words according to each co-occurrence frequency and the weights of the hyperlink words;

descendingly sorting the hyperlink words occurred in the hyperlink word list according to the final weights of the hyperlink words, and obtaining K numbers of hyperlink words that are arranged in first; and

adding hyperlinks to the K numbers of hyperlink words, wherein K is a positive integer.

2. The method according to claim 1, the step of generating a characteristic word list by implementing word segmentation processing for each of the texts, comprising:

extracting all unrepeated words obtained by the word segmentation processing as the characteristic words;

or

removing high-frequency words, stop words and low-frequency words from all the unrepeated words obtained by the word segmentation processing, and considering remaining words as the characteristic words.

3. The method according to claim 1, the step of for each of the characteristic words, computing a co-occurrence frequency between each of the characteristic words and each of the hyperlink words, comprising:

for each characteristic word y and each hyperlink word x, calculating the co-occurrence frequency P(x/y) using a formula of;

P(x/y)=a quantity of xy co-occurrences/a quantity of y occurrences;

wherein, the quantity of xy co-occurrences represents the number of texts appearing the characteristic words y and the hyperlink words x in the same time;

the quantity of y occurrences represents the number of texts appearing the characteristic words y;

or,

for each characteristic word y and each hyperlink word x, calculating the co-occurrence frequency P(x/y) using a formula of;

P(x/y)=H(x,y)/I(x,y)=H(x,y)/(H(x) H(y)−

H(x,y));

wherein, H represents information entropy, I represents mutual information.

4. The method according to claim 1, the step of determining a final weight of each of the hyperlink words according to each co-occurrence frequency and the weights of the hyperlink words, comprising:

for each hyperlink word H, calculating the final weight WH′

of the hyperlink word H;

5. A device for adding hyperlinks to hyperlink words in a text, comprising:

a preprocessing module, configured to create a hyperlink word list in advance, collect a variety of texts, generate a characteristic word list by implementing word segmentation processing for each of the texts, and for each of the characteristic words computer a co-occurrence frequency between each characteristic word list and each hyperlink word, wherein the hyperlink word list comprising a plurality of hyperlink words and the characteristic word list comprising a plurality of characteristic words;

the preprocessing module, further configured to respectively determine an inverse document frequency (IDF) value for each characteristic word, wherein the IDF value is calculated by obtaining a quotient by a quantity of the variety of texts collected divided by a quantity of texts appearing the characteristic words, and calculating a logarithm of the quotient;

an adding module, configured to consider each text to be added a hyperlink as a text X, and process the text by following steps;

carrying out the word segmentation processing to the text X, and obtaining a segmentation result;

extracting the hyperlink words occurred in the hyperlink word list and the characteristic words occurred in the characteristic word list from the segmentation result;

computing a weight of each of the hyperlink words that are occurred in the hyperlink word list, and computing a weight of each of the characteristic words that are occurred in the characteristic word list;

determining a final weight of each of the hyperlink words according to each co-occurrence frequency and the weights of the hyperlink words;

descendingly sorting the hyperlink words occurred in the hyperlink word list according to the final weights of the hyperlink words, and obtaining K numbers of hyperlink words that are arranged in first; and

adding hyperlinks to the K numbers of hyperlink words, wherein, K is a positive integer;

the adding module comprising a processing sub-unit module configured to extract the hyperlink words occurred in the hyperlink word list and the characteristic words occurred in the characteristic word list from the segmentation result;

for each hyperlink word H, calculate the weight WH of the hyperlink word H;

WH=TFH*IDFH, wherein TFH represents TF (term frequency) value of the hyperlink word H and TFH refers to the quantity of the hyperlink word H appearing in the text X, and IDFH represents IDF value of the hyperlink word H;

for each characteristic word F, calculate the weight WH of the extracted characteristic word F;

WF=TFF*IDFF;

wherein TFF represents TF value of the characteristic word F, and IDFF represents IDF value of the characteristic word F.

6. The device according to claim 5, wherein the preprocessing module, comprises:

a first processing unit, configured to create a hyperlink word list;

a second processing unit, configured to collect a variety of texts, generate a characteristic word list by implementing word segmentation processing for each of the texts, and compute the co-occurrence frequency between each characteristic word and each hyperlink word.

7. The device according to claim 6, wherein the second processing unit comprises:

a first processing sub-unit, configured to collect a variety of texts; and

a second processing sub-unit, configured to carry out the word segmentation processing for each of the texts, and extract all unrepeated words obtained by the word segmentation processing as the characteristic words to form a characteristic word list;

or, remove high-frequency words, stop words and low-frequency words from all the unrepeated words obtained by the word segmentation processing, and consider remaining words as the characteristic words to form a characteristic word list;

for each characteristic word y and each hyperlink word x, calculating the co-occurrence frequency P(x/y) using a formula of;

P(x/y)=a quantity of xy co-occurrences/a quantity of y occurrences;

wherein, the quantity of xy co-occurrences represents the number of texts appearing the characteristic words y and the hyperlink words x in the same time;

the quantity of y occurrences represents the number of texts appearing the characteristic words y;

or,

for each characteristic word y and each hyperlink word x, calculating the co-occurrence frequency P(x/y) using a formula of;

P(x/y)=H(x,y)/l(x,y)=H(x,y)/(H(x) H(y)−

H(x,y)), wherein, H represents information entropy;

I represents mutual information.

8. The device according to claim 7, wherein the adding module comprises:

a third processing unit, configured to carry out the word segmentation processing to the text X and obtain a segmentation result; and

a fourth processing unit, configured to extract the hyperlink words occurred in the hyperlink word list and the characteristic words occurred in the characteristic word list from the segmentation result, compute a weight of each of the hyperlink words that are occurred in the hyperlink word list, and compute a weight of each of the characteristic words that are occurred in the characteristic word list, and determine the final weight of each of the hyperlink word according to each co-occurrence frequency and the weights of the hyperlink words; and

a fifth processing unit, configured to descendingly sort the hyperlink words occurred in the hyperlink word list according to the final weights of the hyperlink words, and obtain K numbers of hyperlink words that are arranged in first, and add hyperlinks to the K numbers of hyperlink words, wherein, K is a positive integer.

9. The device according to claim 8, wherein the fourth processing unit comprises:

a fourth processing sub-unit, configured to, for each hyperlink word H, calculate the final weight WH′

of the hyperlink word H;

10. A non-transitory computer readable storage medium, storing one or more programs for execution by one or more processors of a computer having a display, the one or more programs comprising instructions for:

creating a hyperlink word list in advance, the hyperlink word list comprising a plurality of hyperlink words;

collecting a variety of texts, and generating a characteristic word list by implementing word segmentation processing for each of the texts, the characteristic word list comprising a plurality of characteristic words;

respectively determining an IDF (inverse document frequency) value for each characteristic word after generating a characteristic word list by implementing word segmentation processing for each of the texts, wherein the IDF value is calculated by following processes;

obtaining a quotient by a quantity of the variety of texts collected divided by a quantity of texts appearing the characteristic word, and calculating a logarithm of the quotient;

for each of the characteristic words, computing a co-occurrence frequency between each of the characteristic words and each of the hyperlink words;

considering each text to be added a hyperlink as a text X, and processing the text X by following steps;

carrying out the word segmentation processing to the text X, and obtaining a segmentation result;

extracting the hyperlink words occurred in the hyperlink word list and the characteristic words occurred in the characteristic word list from the segmentation result;

computing a weight of each of the hyperlink words that are occurred in the hyperlink word list, and computing a weight of each of the characteristic words that are occurred in the characteristic word list, which comprises;

for each hyperlink word H, calculating the weight WH of the hyperlink word H;

WH=TFH*IDFH;

wherein, TFH represents TF (term frequency) value of the hyperlink word H and TFH refers to the quantity of the hyperlink word H appearing in the text X, and IDFH represents IDF value of the hyperlink word H;

for each characteristic word F, calculating the weight WH of the characteristic word F;

WF=TFF*IDFF;

wherein, TFF represents TF value of the characteristic word F, and IDFF represents IDF value of the characteristic word F;

determining a final weight of each of the hyperlink words according to each co-occurrence frequency and the weights of the hyperlink words;

descendingly sorting the hyperlink words occurred in the hyperlink word list according to the final weights of the hyperlink words, and obtaining K numbers of hyperlink words that are arranged in first; and

adding hyperlinks to the K numbers of hyperlink words, wherein K is a positive integer.