Data integration and knowledge management solution

• US 7,428,517 B2
• Filed: 02/27/2003
• Issued: 09/23/2008
• Est. Priority Date: 02/27/2002
• Status: Active Grant

• Current Assignee: InterSystems Corporation
• Original Assignee: Dirk Medard Helena Van Hyfte, Michaël Rik Frans Brands
• Inventors: Van Hyfte, Dirk Medard Helena, Brands, Michaël Rik Frans
• Primary Examiner(s): Holmes; Michael B.
• Application Number:
  US10/377,146
• Publication Number:
  US 20040010483A1
• Time in Patent Office: 2,035 Days
• Field of Search: 706/45
• US Class Current: 706/45
• CPC Class Codes: G06F 16/367 Ontology
• CPC Class Codes: G06F 40/35 Discourse or dialogue representation

Abstract

A multi-dimensional modeling technique and system for knowledge based management, comprises a perceptual engine of relations and concepts; a logical engine of relations and concepts; an inference engine of relations and concepts. The engines operatively connect to a data analysis engine for analyzing an information source, such as a text, sound, video, images, measurement and control data as to its information and knowledge content.

Claims

1. A computer controlled method for identifying concepts and relations in different types of data pertaining to at least one application domain, said data including at least one of a group consisting of linguistic data, image data, video data, sound data, control data, measurement data, olfactive and tactile data, said method comprising the steps of:

defining concepts associated with an application domain, a concept specifies an amount of information relating to said application domain and is expressed in a logical dimension and a perceptual dimension;

defining relations between said concepts, a relation specifies an informative association between concepts of said application domain in at least one of said logical dimension and said perceptual dimension;

defining concept-relation-concept (CRC) patterns for said application domain, in said logical and perceptual dimension;

acquiring data pertaining to said application domain; and

identifying concepts, relations and CRC patterns in said acquired data in said logical and perceptual dimensions using said concepts, relations and CRC-patterns defined for said application domain when performing a service selected from the group consisting of a data categorization service, a data visualization service, a data distribution service, a decision support service, a data search service, a data summarization service, a data translation service, and a data transformation service.

2. A method according to claim 1, wherein said concepts relations and CRC-patterns are defined in an inference dimension specific for an application domain.

3. A method according claim 1, wherein a CRC-pattern is characterized by having a hierarchical structure defined by said relation and a master concept and a slave concept.

4. A method according to claim 1, wherein said logic dimension comprises a domain specific implicit or explicit logical feature set and wherein said concepts and relations are expressed as subsets of said logical feature set and wherein validity is assessed of said defined CRC-patterns in said logical dimension.

5. A method according to claim 1, wherein said perceptual dimension comprises a domain specific implicit or explicit perceptual feature set and wherein said concepts and relations are expressed as subsets of said perceptual feature set and wherein validity is assessed of said defined CRC-patterns in said perceptual dimension.

6. A method according to claim 1, wherein said concepts relations and CRC-patterns are defined in an inference dimension specific for an application domain, and said inference dimension comprises compositional concepts being domain specific combinations of CRC-patterns in said logical dimension and wherein domain specific combinations are defined of at least one CRC-pattern in said logical dimension and a specific set of perceptual features other than concepts defined in said perceptual dimension and other than relations defined in said perceptual dimension, and wherein validity of said combinations and compositional concepts is assessed.

7. A method according to claim 1, wherein a group of boolean, logical, spatial, temporal and evaluative states is defined, a state being a modifier, refining the information value of a CRC-pattern and having no information value independent from CRC-patterns and wherein states are associated with selected ones of said CRC-patterns, compositional concepts and combinations of CRC-patterns.

8. A method according to claim 1, wherein inference pathways are defined combining selected CRC-patterns, compositional concepts, combinations of CRC-patterns and compositional concepts having associated states, and wherein valid sequences of states associated to a pathway are defined.

9. A method according to claim 1, wherein data pertaining to an application domain is analyzed following the steps of:

identifying in said data concepts and relations as defined in said logical and perceptual dimensions;

forming CRC-patterns in said logical and perceptual dimensions of said concepts and relations identified in said data, and

assessing said CRC-patterns as to their validity in said logical and perceptual dimensions of an application domain.

10. A method according to claim 9, further comprising the steps of:

forming and assessing domain specific combinations of CRC-patterns in said logical dimension, and

assessing combinations of CRC-patterns using domain specific combinations of at least one CRC-pattern in said logical dimension and a specific set of perceptual features other than concepts defined in said perceptual dimension and other than relations defined in said perceptual dimension.

11. A method according to claim 9, wherein said concepts relations and CRC-patterns are defined in an inference dimension specific for an application domain, further comprising the steps of:

forming and building inference pathways by combining selected CRC-patterns, compositional concepts and combinations of CRC-patterns and compositional concepts having associated states;

forming a sequence of states by combining individual states associated with CRC-patterns and compositional concepts in an inference pathway, and

assessing said sequence of states according to valid sequences of states associated with said pathway.

12. A method according to claim 1, wherein new concepts and relations other than said defined concepts and relations are established from said data and said new concepts and relations are matched with defined concepts and relations based on logical and perceptual feature convergence and difference in a domain.

13. A method according to claim 1, wherein application domains are related to each other according to convergence of their feature sets in said logical and perceptual dimensions, and of their definitions of said concepts, relations and CRC-patterns in said logical and perceptual dimensions and wherein data pertaining to different application domains are combined in accordance with said relatedness of said domains.

14. A method according to claim 1, wherein new concepts and relations are combined with each other and with defined concepts and relations defining new CRC-patterns and wherein said new CRC-patterns are assessed according to their compliance with logical and perceptual feature sets in a domain.

15. A method according to claim 1, wherein selected new CRC-patterns are added to said defined CRC-patterns for said application domain.

16. A method according to claim 1, wherein said concepts relations and CRC-patterns are defined in an inference dimension specific for an application domain, and wherein assessed CRC-patterns, assessed compositional concepts and assessed combinations of CRC-patterns are combined into clusters expressed in said logical, perceptual and inference dimensions representing said data, by analyzing information parts of said data using concepts and relations common to said CRC-patterns, compositional concepts and combinations of CRC-patterns.

17. A method according to claim 16, wherein new CRC-patterns are added to clusters based on concepts and relations common to said clusters.

18. A method according to claim 16, wherein clusters are combined into a scheme representing said data based on concepts and relations common to said clusters.

19. A method according to claim 16, wherein clusters are combined into a scheme representing said data based on concepts and relations common to said clusters and wherein said scheme is arranged according to at least one of cluster length, frequency of appearance of clusters, frequency of appearance of concepts, frequency of appearance of relations, frequency of appearance of concepts and relations in a cluster and information value of concepts, relations, and combinations of concepts and relations in CRC-patterns and clusters.

20. A method according to claim 1, wherein different data types are associated in a formal and informative manner based on said definitions of concepts, relations and CRC-patterns in said logical, perceptual and inference dimensions.

21. A method according to claim 1, comprising the steps of:

defining at least one field of interest consisting of selected CRC-patterns;

assigning to each of said CRC-patterns an information value typical for a field of interest

assessing analyzed data as to its compliance with said field of interest by comparing CRC-patterns representing said analyzed data and said field of interest, providing a compliance factor, and

calculating a field correlation factor using said information value and said compliance factor.

22. A method according to claim 21, wherein said correlation factor is calculated for a plurality of different fields of interest.

23. A method according to claim 1, comprising the steps of:

defining at least one user profile consisting of selected CRC-patterns;

assigning to each of said CRC-patterns an information value typical for a field of interest

assessing analyzed data as to its compliance with said user profile by comparing CRC-patterns representing said analyzed data and said field of interest, providing a compliance factor, and

calculating a user correlation factor using said information value and said compliance factor.

24. A method according to claim 23, wherein said user profile is either one of a subset and a superset of either one of at least one field of interest and at least one application domain.

25. A method according to claim 23, wherein said correlation factor is calculated for a plurality of different user profiles.

26. A method according to claims 21, wherein analyzed data are categorized in accordance with said field correlation factor and distributed according to said user correlation factor.

27. A method according to claim 1, wherein said concepts relations and CRC-patterns are defined in an inference dimension specific for an application domain, and wherein an application domain is visualized by forming clusters of concepts and relations defined in said logical, perceptual and inference dimensions and associated states.

28. A method according to claim 27, wherein schemes of clusters are combined with said domain visualizations based on concepts and relations common to said schemes and domain visualizations.

29. A method according to claim 1, wherein presence of structures of co-occurring CRC-patterns in analyzed data is detected, and wherein recurring sequences of states associated with said structures are assessed.

30. A method according to claim 1, wherein detected structures of co-occurring CRC-patterns and sequences of associated states are combined into inference pathways and inference pathways are defined as processes if said sequence of associated states determines a dependancy between concepts at both ends of a pathway.

31. A method according to claim 30, wherein procedures are defined as specific combinations of processes and determined by specific sequences of associated states.

32. A method according to claim 30, wherein processes and procedures are used for categorizing and distributing analyzed data.

33. A method according to claim 30, wherein at least one procedure and at least one trigger are combined into a task, a trigger being any of a specific perceptual concept and a specific state, and wherein said task is performed by executing said at least one procedure in response to said at least one trigger.

34. A method according to claim 1, wherein data are searched in function of a query comprising at least one of a specified concept, a specified combination of a concept and a relation, a specified CRC-pattern, a specified compositional concept, a specified inference pathway, a specified process, a specified procedure and specified combinations thereof in at least one of said logical, perceptual and inference dimensions.

35. A method according to claim 34, wherein a broad search is based on the definition of all concepts and relations of said query in said logical dimension and implies all perceptual features of said concepts and said relations and wherein a narrow search is based on at least one specified feature of all concepts and relations of said query pertaining to the definition of said concepts and said relations in said perceptual dimension and a combined search is based on the definition in said logical dimension of at least one of said concepts and relations of said query and of said definition of at least one specified feature in said perceptual dimension of at least one of said other concepts and relations of said query.

36. A method according to claim 34, wherein a complete set of boolean search strings describing said queries is generated.

37. A method according to claim 1, wherein a summary of analyzed data is generated based on clusters of CRC-patterns, selected in accordance with a specified level of information value, of concepts and relations in an analysis scheme of said data, and wherein said selected clusters are adapted into a correct representation in at least one of said data types comprised in said perceptual dimension.

38. A method according to claim 1, wherein data, analyzed into clusters of CRC-patterns, of a first data type comprised in said perceptual dimension is transformed in to a correct representation in at least one second data type comprised in said perceptual dimension, by replacing analyzed feature sets of said first data type defining said clusters with features sets of said perceptual dimension pertaining to said second data type describing the same clusters and referring to the same concepts and relations defined in said logical dimension.

39. A method according to claim 38, wherein analyzed data of the linguistic data type, the video data type and the sound data type belonging at least partially to at least one natural language covered by the feature set defining said perceptual dimension are translated into a correct representation in another natural language covered by the feature set defining said perceptual dimension.

40. A method according to claim 1, comprising the steps of:

converting data of one of the types comprised in said perceptual dimension having any file format into a format for processing said data type,

detecting information parts within said converted data

removing non-informative data from said information parts by assessing them against predefined non-informative data,

determining frequency of occurrence of informative data and rerating said frequency of certain predefined informative data, providing rated information blocks, and

organizing said rated information blocks into data for analyzing same using said concepts and said relations.

84. A computer program product comprising a computer program arranged for performing a method according to claim 1, if loaded into a memory of an electronic processing device.

85. A computer program product according to claim 84, comprising any of a group of data carrying devices including floppy discs, CO-roms, DVDs, tapes, memory stick devices, zip drives, flash memory cards, remote access devices, hard disks, semi-conductor memory devices, programmable semi-conductor memory devices, optical disks, magneto-optical data storage devices, and ferro electric data storage devices.

41. A computer controlled system for identifying concepts and relations in different types of data pertaining to at least one application domain, said data including at least one of a group consisting of linguistic data, image data, video data, sound data, control data, measurement data, olfactive and tactile data, said system comprising:

a concepts repository comprising concepts associated with an application domain, a concept specifying an amount of information relating to said application domain and expressed in a logical dimension and a perceptual dimension;

a relations repository comprising relations between said concepts, a relation specifying an informative association between concepts of said application domain in at least one of said logical dimension and said perceptual dimension;

a CRC-pattern repository comprising CRC-patterns defined for said application domain, in said logical and perceptual dimension, and

an analysis engine for acquiring data pertaining to said application domain and arranged for identifying concepts, relations and CRC patterns in said acquired data in said logical and perceptual dimensions using said concepts, relations and CRC-patterns defined for said application domain when performing a service selected from the group consisting of a data categorization service, a data visualization service, a data distribution service, a decision support service, a data search service, a data summarization service, a data translation service, and a data transformation service.

42. A system according to claim 41, having an inference repository comprising said concepts relations and CRC-patterns expressed in an inference dimension specific for an application domain.

43. A system according to claim 41, wherein said concept repository and relation repository comprise a domain specific logical feature set and wherein said concepts and relations are expressed as subsets of said logical feature set.

44. A system according to claim 41, wherein said analysis engine is arranged for assessing validity of said CRC-patterns in said logical dimension.

45. A system according to claim 41, wherein said concept repository and relation repository comprise a domain specific perceptual feature set, and wherein said concepts and relations are expressed as subsets of said perceptual feature set.

46. A system according to claim 45, wherein said analysis engine is arranged for assessing validity of said CRC-patterns in said perceptual dimension.

47. A system according to claim 41, having an inference repository comprising said concepts relations and CRC-patterns expressed in an inference dimension specific for an application domain, and wherein said inference repository comprises compositional concepts defined as domain specific combinations of CRC-patterns in said logical dimension and comprising domain specific combinations of at least one CRC-pattern in said logical dimension and a specific set of perceptual features other than concepts defined in said perceptual dimension and other than relations defined in said perceptual dimension.

48. A system according to claim 47, wherein said analysis engine is arranged for assessing validity of said combinations and compositional concepts in said inference dimension.

49. A system according to claim 41, having an inference repository comprising said concepts relations and CRC-patterns expressed in an inference dimension specific for an application domain, and wherein at least one state—

a state being a modifier, refining the information value of a CRC-pattern and having no information value independent from CRC-patterns—

of a group of boolean, logical, spatial, temporal and evaluative states is associated with selected ones of said CRC-patterns, compositional concepts and combinations of CRC-patterns of said inference repository.

50. A system according to claim 49, wherein said inference repository comprises inference pathways consisting of combinations of selected CRC-patterns, compositional concepts, combinations of CRC-patterns and compositional concepts having associated states, and comprising valid sequences of states associated to a pathway.

51. A system according to claim 41, wherein said analysis engine comprises:

a logical engine arranged for identifying in said data concepts and relations defined in said logical dimension;

a perceptual engine arranged for identifying in said data concepts and relations defined in said perceptual dimension, and

CRC-pattern assessment means arranged for identifying said CRC-patterns as to their definitions in said logical and perceptual dimensions of an application domain.

52. A system according to claim 51, further comprising a logical exclusions repository comprising combinations of concepts and relations providing invalid CRC-patterns and wherein said analysis engine comprises invalid logical CRC-pattern detection means excluding invalid CRC-patterns.

53. A system according to claim 51, wherein said analysis engine comprises an inference engine arranged for:

forming and assessing domain specific combinations of CRC-patterns in said logical dimension, and

assessing defined compositional concepts and assessing combinations of CRC-patterns using domain specific combinations of at least one CRC-pattern in said logical dimension and a specific set of perceptual features other than concepts defined in said perceptual dimension other than relations defined in said perceptual dimension.

54. A system according to claim 53, wherein said inference engine further comprises means arranged for:

forming and building inference pathways by combining selected CRC-patterns, compositional concepts and combinations of CRC-patterns and compositional concepts having associated states;

forming a sequence of states by combining individual states associated with CRC-patterns and compositional concepts in an inference pathway, and

assessing said sequence of associated states according to valid sequences of states associated with said pathway.

55. A system according to claim 41, comprising means for:

establishing new concepts and relations other than said defined concepts and relations in said data, and matching said new concepts and relations with defined concepts and relations based on logical and perceptual feature convergence and difference in a domain.

56. A system according to claim 55, wherein said means for matching new concepts and relations are arranged for:

combining said new concepts and relations with each other and with defined concepts and relations to define new CRC-patterns, and

assessing said new CRC-patterns according to their compliance with logical and perceptual feature sets in a domain.

57. A system according to claim 41, comprising means to add selected new CRC-patterns to said defined CRC-patterns in said CRC-patterns repository.

58. A system according to claim 41, comprising a categorization engine arranged for:

defining at least one field of interest consisting of selected CRC-patterns;

assigning to each of said CRC-patterns an information value typical for a field of interest

assessing analyzed data as to its compliance with said field of interest by comparing CRC-patterns representing said analyzed data and said field of interest, providing a compliance factor, and

calculating a field correlation factor using said information value and said compliance factor.

59. A system according to claim 58, wherein said categorization engine is arranged for calculating said correlation factor for a plurality of different fields of interest.

60. A system according to claim 58, wherein said categorization engine is arranged for:

defining at least one user profile consisting of selected CRC-patterns;

assigning to each of said CRC-patterns an information value typical for a field of interest;

assessing analyzed data as to its compliance with said user profile by comparing CRC-patterns representing said analyzed data and said field of interest, providing a compliance factor, and

calculating a user correlation factor using said information value and said compliance factor.

61. A system according to claim 58, wherein said categorization engine is arranged for calculating said correlation factor for a plurality of different user profiles.

62. A system according to claim 58, wherein said categorization engine is arranged for categorizing said analyzed data in accordance with said field correlation factor.

63. A system according to claim 58, comprising a distribution engine arranged for distributing said analyzed data in accordance with said user correlation factor.

64. A system according to claim 41, having an inference repository comprising said concepts relations and CRC-patterns expressed in an inference dimension specific for an application domain, and comprising a vizualisation engine arranged for vizualising an application domain by forming clusters of concepts and relations defined in said logical, perceptual and inference dimensions and associated states.

65. A system according to claim 53, wherein said vizualisation engine is arranged for combining assessed CRC-patterns, assessed compositional concepts and assessed combinations of CRC-patterns into clusters expressed in said logical, perceptual and inference dimensions representing said data, by analyzing information parts of said data using concepts and relations common to said CRC-patterns, compositional concepts and combinations of CRC-patterns.

66. A system according to claim 64, wherein said vizualisation engine is arranged for adding new CRC-patterns to clusters based on concepts and relations common to said clusters.

67. A system according to claim 64, wherein said vizualisation engine is arranged for combining clusters into a scheme representing said data based on concepts and relations common to said clusters.

68. A system according to claim 64, wherein said vizualisation engine is arranged for combining clusters into a scheme representing said data based on concepts and relations common to said clusters, for composing said scheme in accordance with at least one of cluster length, frequency of appearance of clusters, frequency of appearance of concepts, frequency of appearance of relations, frequency of appearance of concepts and relations in a cluster and information value of concepts, relations, and combinations of concepts and relations in CRC-patterns and clusters.

69. A system according to claim 64, wherein said vizualisation engine is arranged for combining said schemes of clusters with said domain vizualisations based on concepts and relations common to said schemes and domain visualizations.

70. A system according to claim 41, comprising a process analysis engine arranged for assessing presence of structures of co-occurring CRC-patterns in analyzed data, and for assessing recurring sequences of states associated with said structures.

71. A system according to claim 70, wherein said process analysis engine is arranged for detecting structures of co-occurring CRC-patterns and for combining sequences of associated states into inference pathways, inference pathways are defined as processes if said sequence of associated states determines a dependency between concepts at both ends of a pathway.

72. A system according to claim 70, wherein said process analysis engine is arranged for detecting procedures as specific combinations of processes and specific sequences of associated states.

73. A system according to claim 70, wherein said categorization engine is arranged for categorizing analyzed data according to said processes and said procedures.

74. A system according to claim 70, wherein said distribution engine is arranged for distributing analyzed data according to said processes and said procedures.

75. A system according to claim 74, comprising a decision advice engine arranged for combining at least one procedure and at least one trigger into a task, a trigger being any of a specific perceptual concept and a specific state, and for performing said task by executing said at least one procedure in response to said at least one trigger.

76. A system according to claim 41, comprising a search engine arranged for searching data in function of a query comprising at least one of a specified concept, a specified combination of a concept and a relation, a specified CRC-pattern, a specified compositional concept, a specified inference pathway, a specified process, a specified procedure and specified combination thereof in at least one of said logical, perceptual and inference dimensions.

77. A system according to claim 76, wherein said search engine is arranged for performing a broad search based on the definition of all concepts and relations of said query in said logical dimension and implies all perceptual features of said concepts and said relations, a narrow search based on at least one specified feature of all concepts and relations of said query pertaining to the definition of said concepts and said relations in said perceptual dimension, and a combined search based on the definition in said logical dimension of at least one of the concepts and relations of said query and of the definition of at least one specified feature in said perceptual dimension of at least one of the other concepts and relations of said query.

78. A system according to claim 76, wherein said search engine is arranged for generating a complete set of boolean search strings describing said queries.

79. A system according to claim 41, comprising a summarizing engine arranged for generating a summary of analyzed data based on clusters of CRC-patterns, selected in accordance with a specified level of information value, of concepts and relations in an analysis scheme of said data and for adapting said selected clusters into a correct representation in at least one of the data types comprised in said perceptual dimension.

80. A system according to claim 41, comprising a translation engine arranged for transforming data, analyzed into clusters of CRC-patterns, of a first data type comprised in said perceptual dimension into a correct representation in at least one second data type comprised in said perceptual dimension, by replacing analyzed feature sets of said first data type defining said clusters with features sets of said perceptual dimension pertaining to said second data type describing the same clusters and referring to the same concepts and relations defined in said logical dimension.

81. A system according to claim 80, wherein said translation engine is arranged for translating analyzed data of the linguistic data type, the video data type and the sound data type belonging at least partially to at least one natural language covered by the feature set defining said perceptual dimension into a correct representation in an other natural language covered by the feature set defining said perceptual dimension.

82. A system according to claim 41, comprising a plurality of preprocessing engines, each preprocessing engine being arranged for preprocessing one of said data types defined in said perceptual dimension by:

converting data of one of said data types comprised in said perceptual dimension having any file format into a format for processing said data type;

detecting information parts within said converted data;

removing non-informative data from said information parts by assessing them against predefined non-informative data;

determining frequency of occurrence of informative data and rerating said frequency of certain predefined informative data, providing rated information blocks, and

organizing said rated information blocks into data for analyzing same using said concepts and said relations.

83. A system according to claim 41, comprising an updating engine arranged for updating said concepts repository, said relations repository, said CRC-patterns repository, said logical exclusions repository and said inference repository by adding new concepts, relations, CRC-patterns, compositional concepts, combinations of CRC-patterns, inference pathways, processes and procedures.