System and method for generating vocabulary from network data

• US 8,489,390 B2
• Filed: 09/30/2009
• Issued: 07/16/2013
• Est. Priority Date: 09/30/2009
• Status: Active Grant

• Current Assignee: Cisco Technology, Inc. (Cisco Systems, Inc.)
• Original Assignee: Cisco Technology, Inc. (Cisco Systems, Inc.)
• Inventors: Arumugam, Thangavelu, Gannu, Satish K., Mihailovici, Virgil N., Malegaonkar, Ashutosh A., Posse, Christian, Sambhus, Sonali M., Walia, Nitasha, Zhang, Kui
• Primary Examiner(s): JACKSON, JAKIEDA R
• Application Number:
  US12/571,390
• Publication Number:
  US 20110077936A1
• Time in Patent Office: 1,385 Days
• Field of Search: 704/200.1
• US Class Current: 704/200.1
• CPC Class Codes: G06F 16/9535 Search customisation based on user profiles and personalisation
• CPC Class Codes: G06F 40/117 Tagging; Marking up details of markup languages G06F40/143; Designating a block; Setting of attributes style sheets, e.g. eXtensible Stylesheet Language Transformation [XSLT], G06F40/154
• CPC Class Codes: G06F 40/289 Phrasal analysis, e.g. finite state techniques or chunking

Abstract

A method is provided in one example and includes receiving data propagating in a network environment and separating the data into one or more fields. At least some of the fields are evaluated in order to identify nouns and noun phrases within the fields. The method also includes identifying selected words within the nouns and noun phrases based on a whitelist and a blacklist. The whitelist includes a plurality of designated words to be tagged and the blacklist includes a plurality of rejected words that are not to be tagged. A resultant composite is generated for the selected nouns and noun phrases that are tagged. The resultant composite is incorporated into the whitelist if the resultant composite is approved.

Claims

1. A method, comprising:

receiving data propagating in a network environment;

sorting the data into a first group and a second group, wherein the first group includes Joint Photographic Experts Group (JPEG) compressed data, and wherein the first group is ignored;

separating the data in the second group into one or more fields;

evaluating, using a processor, at least some of the fields in order to identify nouns and noun phrases within the fields;

identifying selected words within the nouns and noun phrases based on a whitelist and a blacklist, wherein the whitelist includes a plurality of designated words to be tagged and the blacklist includes a plurality of rejected words that are not to be tagged;

dropping the data if certain words in the data are included in the blacklist;

generating a first resultant composite of selected nouns and noun phrases that are tagged;

identifying selected words within the first resultant composite of selected nouns and noun phrases based on a list of administrator stop words;

removing the identified selected words to create a second resultant composite of selected nouns and noun phrases;

presenting the second resultant composite of selected nouns and noun phrases to an administrator; and

incorporating selected nouns and noun phrases from the second resultant composite into the whitelist if the selected nouns and noun phrases are approved by the administrator.

2. The method of claim 1, wherein the fields include a title field, a concept field, a text field, and an authorship field.

3. The method of claim 2, further comprising:

splitting a document associated with the data into a first segment that includes the concept field and a second segment that includes fields other than the concept field.

4. The method of claim 1, further comprising:

removing one or more words in the data based on a configured listing of stop words, which are identified in the data.

5. The method of claim 1, further comprising:

suggesting words for inclusion into the whitelist or the blacklist based on an aggregation of data associated with a plurality of end users.

6. The method of claim 1, further comprising:

identifying topical areas within the data; and

providing the topical areas to an administrative entity for inclusion into the first resultant composite.

7. The method of claim 1, further comprising:

filtering the data based on a document type associated with the data.

8. Logic encoded in one or more non-transitory media that includes code for execution and when executed by a processor is operable to perform operations comprising:

receiving data propagating in a network environment;

sorting the data into a first group and a second group, wherein the first group includes Joint Photographic Experts Group (JPEG) compressed data, and wherein the first group is ignored;

separating the data in the second group into one or more fields;

evaluating at least some of the fields in order to identify nouns and noun phrases within the fields;

identifying selected words within the nouns and noun phrases based on a whitelist and a blacklist, wherein the whitelist includes a plurality of designated words to be tagged and the blacklist includes a plurality of rejected words that are not to be tagged;

dropping the data if certain words in the data are included in the blacklist;

generating a first resultant composite of selected nouns and noun phrases that are tagged;

identifying selected words within the first resultant composite of selected nouns and noun phrases based on a list of administrator stop words;

removing the identified selected words to create a second resultant composite of selected nouns and noun phrases;

presenting the second resultant composite of selected nouns and noun phrases to an administrator; and

incorporating selected nouns and noun phrases from the second resultant composite into the whitelist if the selected nouns and noun phrases are approved by the administrator.

9. The logic of claim 8, wherein the fields include a title field, a concept field, a text field, and an authorship field.

10. The logic of claim 9, the processor being further operable to perform operations comprising:

splitting a document associated with the data into a first segment that includes the concept field and a second segment that includes fields other than the concept field.

11. The logic of claim 8, the processor being further operable to perform operations comprising:

removing one or more words in the data based on a configured listing of stop words, which are identified in the data.

12. The logic of claim 8, the processor being further operable to perform operations comprising:

suggesting words for inclusion into the whitelist or the blacklist based on an aggregation of data associated with a plurality of end users.

13. The logic of claim 8, the processor being further operable to perform operations comprising:

identifying topical areas within the data; and

providing the topical areas to an administrative entity for inclusion into the first resultant composite.

14. The logic of claim 8, the processor being further operable to perform operations comprising:

filtering the data based on a document type associated with the data.

15. An apparatus, comprising:

a memory element;

a processor operable to execute instructions; and

a noun phrase extractor module configured to interface with the memory element and the processor, the noun phrase extractor module being configured to;

receive data propagating in a network environment;

sort the data into a first group and a second group, wherein the first group includes Joint Photographic Experts Group (JPEG) compressed data, and wherein the first group is ignored;

separate the data in the second group into one or more fields;

evaluate at least some of the fields in order to identify nouns and noun phrases within the fields;

identify selected words within the nouns and noun phrases based on a whitelist and a blacklist, wherein the whitelist includes a plurality of designated words to be tagged and the blacklist includes a plurality of rejected words that are not to be tagged;

drop the data if certain words in the data are included in the blacklist;

generate a first resultant composite of selected nouns and noun phrases that are tagged;

identify selected words within the first resultant composite of selected nouns and noun phrases based on a list of administrator stop words;

remove the identified selected words to create a second resultant composite of selected nouns and noun phrases;

present the second resultant composite of selected nouns and noun phrases to an administrator; and

incorporate selected nouns and noun phrases from the second resultant composite into the whitelist if the selected nouns and noun phrases are approved by the administrator.

16. The apparatus of claim 15, wherein the fields include a title field, a concept field, a text field, and an authorship field.

17. The apparatus of claim 16, further comprising:

a document splitter configured to split a document associated with the data into a first segment that includes the concept field and a second segment that includes fields other than the concept field.

18. The apparatus of claim 15, further comprising:

a user interface configured to suggest words for inclusion into the whitelist or the blacklist based on an aggregation of data associated with a plurality of end users.

19. The apparatus of claim 18, wherein topical areas are identified within the data and provided to an administrative entity at the user interface for inclusion into the first resultant composite.

20. The apparatus of claim 15, further comprising:

a document filter configured to filter the data based on a document type associated with the data.

21. A system, comprising:

a network element that includes a memory element and a processor operable to execute instructions, wherein the network element is configured to;

receive data propagating in a network environment;

sort the data into a first group and a second group, wherein the first group includes Joint Photographic Experts Group (JPEG) compressed data, and wherein the first group is ignored;

separate the data in the second group into one or more fields;

evaluate at least some of the fields in order to identify nouns and noun phrases within the fields;

identify selected words within the nouns and noun phrases based on a whitelist and a blacklist, wherein the whitelist includes a plurality of designated words to be tagged and the blacklist includes a plurality of rejected words that are not to be tagged;

drop the data if certain words in the data are included in the blacklist;

generate a first resultant composite of selected nouns and noun phrases that are tagged;

identify selected words within the first resultant composite of selected nouns and noun phrases based on a list of administrator stop words;

remove the identified selected words to create a second resultant composite of selected nouns and noun phrases;

present the second resultant composite of selected nouns and noun phrases to an administrator;

incorporate selected nouns and noun phrases from the second resultant composite into the whitelist if the selected nouns and noun phrases are approved by the administrator; and

maintain a repository that includes the second resultant composite, wherein the repository is configured to receive one or more search queries associated with designated subject areas.

22. The system of claim 21, wherein the network element is configured to:

remove one or more stop words associated with the data propagating in the network environment, wherein the stop words are included in a stop word list.

23. The system of claim 21, wherein the fields include a title field, a concept field, a text field, and an authorship field.

24. The system of claim 21, wherein the network element is configured to:

reconstruct documents carried by an email protocol.

25. The system of claim 21, wherein the network element is configured to:

recognize an application associated with a document relating to the data propagating in the network.

26. The system of claim 21, wherein the network element is configured to:

split a document associated with the data into a first segment that includes the concept field and a second segment that includes fields other than the concept field.

27. The system of claim 21, wherein the network element is configured to:

incorporate the selected nouns and noun phrases into the whitelist if the selected nouns and noun phrases are approved.

28. The system of claim 21, wherein the network element is configured to:

communicate a tagged document over a secure protocol to a next destination.

29. The system of claim 21, wherein the network element is configured to:

suggest words for inclusion into the whitelist or the blacklist based on an aggregation of data associated with a plurality of end users.