Method and System for Unified Information Representation and Applications Thereof

• US 20120233127A1
• Filed: 03/10/2011
• Published: 09/13/2012
• Est. Priority Date: 03/10/2011
• Status: Active Grant

• Current Assignee: TextWise Company, LLC
• Original Assignee: TextWise Company, LLC
• Inventors: Solmer, Robert, Ruan, Wen
• Granted Patent: US 8,548,951 B2
• Time in Patent Office: Days
• US Class Current: 707/661
• CPC Class Codes: G06F 16/113 Details of archiving lifecycle management in storage systems G06F3/0649; backup systems G06F11/1446
• CPC Class Codes: G06F 16/3347 using vector based model
• CPC Class Codes: G06F 16/93 Document management systems
• CPC Class Codes: G06F 2212/454 Vector or matrix data

Abstract

Method, system, and programs for information search and retrieval. A query is received and is processed to generate a feature-based vector that characterizes the query. A unified representation is then created based on the feature-based vector, that integrates semantic and feature based characterizations of the query. Information relevant to the query is then retrieved from an information archive based on the unified representation of the query. A query response is generated based on the retrieved information relevant to the query and is then transmitted to respond to the query.

Claims

1. A method, implemented on a machine having at least one processor, storage, and a communication platform connected to a network for archiving data, comprising the steps of:

receiving data via the communication platform;

analyzing, by a feature extractor, the received data in accordance with at least one model to form a feature-based vector characterizing the data;

generating, by a semantic extractor, a semantic-based representation of the data based on the feature-based vector;

constructing, by a reconstruction unit, a reconstructed feature-based vector based on the semantic-based representation of the data;

comparing, by a discrepancy analyzer, the feature-based vector with the reconstructed feature-based vector;

forming a residual feature-based representation of the data based on one or more features identified in accordance with a result of the comparing;

generating, by a unified representation construction unit, a unified representation for the data based on the semantic-based representation and the residual feature-based representation; and

archiving the data in an information archive based on the unified representation of the data.

2. The method of claim 1, wherein the at least one model includes an information model and a language model, wherein the feature-based vector according to the information model has a plurality of attributes, each of which represents a portion of information contained in the data allocated to an associated feature.

3. The method of claim 1, wherein the semantic extractor and the reconstruction unit are implemented based on an auto-encoder.

4. The method of claim 1, wherein the step of archiving comprises:

computing, by an indexing system, an index value based on the unified representation for the data;

establishing a link between the index value and the data archived in accordance with the unified representation of the data in the information archive.

5. The method of claim 1, further comprising forming a blurred feature-based representation of the data by modifying the feature-based vector based on the reconstructed feature-based vector.

6. The method of claim 5, wherein the step of modifying the feature-based vector comprises:

obtaining a first attribute value from the feature-based vector;

obtaining a second attribute value corresponding to the first attribute from the reconstructed feature-based vector; and

computing a third attribute value as the corresponding attribute value of the blurred feature-based representation of the data based on the first and second attribute values.

7. The method of claim 5, wherein the unified representation for the data is constructed further based on the blurred feature-based representation of the data.

8. A method, implemented on a machine having at least one processor, storage, and a communication platform connected to a network for archiving data, comprising the steps of:

receiving data via the communication platform;

analyzing, by a feature extractor, the received data in accordance with at least one model to form a feature-based vector characterizing the data;

generating, by a semantic extractor, a semantic-based representation of the data based on the feature-based vector;

constructing, by a reconstruction unit, a reconstructed feature-based vector based on the semantic-based representation of the data;

forming a blurred feature-based representation of the data by modifying the feature-based vector based on the reconstructed feature-based vector;

generating, by a unified representation construction unit, a unified representation for the data based on the blurred feature-based representation; and

archiving the data in an information archive based on the unified representation of the data.

9. The method of claim 8, further comprising:

forming a residual feature-based representation of the data based on one or more features identified in accordance with discrepancy between the feature-based vector and the reconstructed feature-based vector; and

incorporating, in the unified representation for the data, the semantic-based representation and the residual feature-based representation.

10. The method of claim 8, wherein the at least one model includes an information model and a language model, wherein the feature-based vector according to the information model has a plurality of attributes, each of which represents a portion of information contained in the data allocated to an associated feature.

11. The method of claim 8, wherein the semantic extractor and the reconstruction unit are implemented based on an auto-encoder.

12. A method, implemented on a machine having at least one processor, storage, and a communication platform connected to a network for search and retrieval of data archived based on a unified representation, comprising the steps of:

obtaining a query via the communication platform;

processing, by a query processor, the query to generate a feature-based vector characterizing the query;

generating, by a query representation generator, a unified representation for the query based on the feature-based vector, wherein the unified representation integrates semantic and feature based characterizations of the query;

retrieving, by a candidate search unit, information relevant to the query from an information archive based on the unified representation for the query;

generating, by a query response generator, a query response based on the information relevant to the query retrieved from the information archive;

transmitting the query response to respond to the query.

13. The method of claim 12, wherein the step of generating the unified representation of the query comprises the steps of:

generating, by a semantic extractor, a semantic-based representation of the query based on the feature-based vector;

constructing, by a reconstruction unit, a reconstructed feature-based vector based on the semantic-based representation of the query;

comparing, by a discrepancy analyzer, the feature-based vector with the reconstructed feature-based vector;

forming a residual feature-based representation of the query based on a result of the comparing; and

generating, by a unified representation construction unit, the unified representation based on the semantic-based representation and the residual feature-based representation.

14. The method of claim 13, wherein the unified representation of the query further includes a blurred feature-based representation generated by modifying the feature-based vector based on the reconstructed feature-based vector.

15. The method of claim 12, wherein the step of retrieving comprises:

generating a first index value based on the unified representation of the query;

identifying a second index value stored in an indexing system of the information archive;

obtaining a group of information items in the information archive that have similar index values; and

selecting the information relevant to the query from the obtained group of information items.

16. A system for generating a unified representation for data, comprising:

a communication platform through which data can be received;

a feature extractor configured for analyzing the received data in accordance with at least one model to form a feature-based vector characterizing the data;

a semantic extractor configured for generating a semantic-based representation of the data based on the feature-based vector;

a reconstruction unit configured for producing a reconstructed feature-based vector based on the semantic-based representation of the data;

a residual feature identifier configured for forming a residual feature-based representation of the data based on one or more residual features identified in accordance with a comparison between the feature-based vector and the reconstructed feature-based vector; and

a unified representation construction unit configured for generating a unified representation for the data based on the semantic-based representation and the residual feature-based representation.

17. The system of claim 16, wherein the at least one model includes an information model and a language model, wherein the feature-based vector built according to the information model has a plurality of attributes, each of which represents a portion of information contained in the data allocated to an associated feature.

18. The system of claim 16, wherein the semantic extractor and the reconstruction unit are implemented based on an auto-encoder.

19. The system of claim 16, further comprising a feature vector blurring unit configured for forming a blurred feature-based representation of the data by modifying the feature-based vector based on the reconstructed feature-based vector.

20. A system for search and retrieval of data archived based on a unified representation, comprising:

a communication platform for obtaining a query and transmitting a query response;

a query processor configured for processing the query to generate a feature-based vector characterizing the query;

a query representation generator configured for generating a unified representation for the query based on the feature-based vector, wherein the unified representation integrates semantic and feature based characterizations of the query;

a candidate search unit configured for retrieving information relevant to the query from an information archive based on the unified representation for the query; and

a query response generator configured for generating the query response based on the information relevant to the query retrieved from the information archive and transmitting the query response to respond to the query.

21. The system of claim 20, further comprising a feature vector blurring unit configured for generating a blurred feature-based representation by modifying the feature-based vector based on the reconstructed feature-based vector.

22. A machine-readable non-transitory medium having information recorded thereon related to data archiving, the information, when read by the machine, causes the machine to perform the following:

receiving data via a communication platform;

analyzing the received data in accordance with at least one model to form a feature-based vector characterizing the data;

generating a semantic-based representation of the data based on the feature-based vector;

constructing a reconstructed feature-based vector based on the semantic-based representation of the data;

comparing the feature-based vector with the reconstructed feature-based vector;

forming a residual feature-based representation of the data based on one or more features identified in accordance with a result of the comparing;

generating a unified representation for the data based on the semantic-based representation and the residual feature-based representation; and

archiving the data in an information archive based on the unified representation of the data.

23. The medium of claim 22, wherein the at least one model includes an information model and a language model, wherein the feature-based vector according to the information model has a plurality of attributes, each of which represents a portion of information contained in the data allocated to an associated feature.

24. The medium of claim 22, wherein the semantic extractor and the reconstruction unit are implemented based on an auto-encoder.

25. The medium of claim 22, wherein the information, when read by the machine, further causes the machine to perform the following:

forming a blurred feature-based representation of the data by modifying the feature-based vector based on the reconstructed feature-based vector; and

incorporating the blurred feature-based representation of the data as part of the unified representation for the data.

26. A machine-readable non-transitory medium having information recorded thereon for data archiving, the information, when read by the machine, causes the machine to perform the following:

receiving data via a communication platform;

analyzing the received data in accordance with at least one model to form a feature-based vector characterizing the data;

generating a semantic-based representation of the data based on the feature-based vector;

constructing a reconstructed feature-based vector based on the semantic-based representation of the data;

forming a blurred feature-based representation of the data by modifying the feature-based vector based on the reconstructed feature-based vector;

generating a unified representation for the data based on the blurred feature-based representation; and

archiving the data in an information archive based on the unified representation of the data.

27. The medium of claim 26, wherein the information, when read by the machine, further causes the machine to perform the following:

forming a residual feature-based representation of the data based on one or more features identified in accordance with discrepancy between the feature-based vector and the reconstructed feature-based vector; and

incorporating, in the unified representation for the data, the semantic-based representation and the residual feature-based representation.

28. The medium of claim 26, wherein the at least one model includes an information model and a language model, wherein the feature-based vector according to the information model has a plurality of attributes, each of which represents a portion of information contained in the data allocated to an associated feature.

29. The medium of claim 26, wherein the semantic extractor and the reconstruction unit are implemented based on an auto-encoder.

30. A machine-readable non-transitory medium having information recorded thereon for information search and retrieval, when read by the machine, causes the machine to perform the following:

obtaining a query via a communication platform;

processing the query to generate a feature-based vector characterizing the query;

generating a unified representation for the query based on the feature-based vector, wherein the unified representation integrates semantic and feature based characterizations of the query;

retrieving information relevant to the query from an information archive based on the unified representation for the query;

generating a query response based on the information relevant to the query retrieved from the information archive; and

transmitting the query response to respond to the query.

31. The medium of claim 30, wherein the step of generating the unified representation of the query comprises the steps of:

generating a semantic-based representation of the query based on the feature-based vector;

constructing a reconstructed feature-based vector based on the semantic-based representation of the query;

comparing the feature-based vector with the reconstructed feature-based vector;

forming a residual feature-based representation of the query based on a result of the comparing; and

generating the unified representation based on the semantic-based representation and the residual feature-based representation.

32. The medium of claim 30, wherein the unified representation of the query further includes a blurred feature-based representation generated by modifying the feature-based vector based on the reconstructed feature-based vector.

33. The medium of claim 30, wherein the step of retrieving comprises:

generating a first index value based on the unified representation of the query;

identifying a second index value stored in an indexing system of the information archive;

obtaining a group of information items in the information archive that have similar index values; and

selecting the information relevant to the query from the obtained group of information items.