SYSTEMS AND METHODS OF DISCOVERING MIXTURES OF MODELS WITHIN DATA AND PROBABILISTIC CLASSIFICATION OF DATA ACCORDING TO THE MODEL MIXTURE

• US 20100042563A1
• Filed: 08/14/2009
• Published: 02/18/2010
• Est. Priority Date: 08/14/2008
• Status: Active Grant

• Current Assignee: United States Department Of Health And Human Services
• Original Assignee: The Navy Chief Of Naval Research Office Of Counsel
• Inventors: Palelu, Aditya Maruti, Livingston, Mark Alan
• Granted Patent: US 8,521,659 B2
• Time in Patent Office: Days
• US Class Current: 706/12
• CPC Class Codes: G06F 18/2321 using statistics or function optimisation, e.g. modelling of probability density functions
• CPC Class Codes: G06F 18/41 Interactive pattern learning with a human teacher
• CPC Class Codes: G06N 20/00 Machine learning

Abstract

Discovering mixtures of models includes: initiating learning algorithms, determining, data sets including a cluster of points in a first region of a domain and a set of points distributed near a first line extending across the domain; inferencing parameters from the cluster and the set of points; creating a description of the cluster of points in the first region of the domain and computing approximations of a first learned mixture model and a second learned mixture model; determining a first and second probability, generating a confidence rating that each point of the cluster of points in the first region of the domain corresponds to the first learned mixture model and generating a confidence rating that each point of the set of points distributed near the first line correspond to the second learned mixture model, thus causing determinations of behavior of a system described by the learned mixture models.

Claims

1. A method, implemented in a computer readable and executable program on a computer processor, of discovering mixtures of models within data and probabilistic classification of data according to model mixtures, the method comprising:

receiving, a request for discovering mixtures of models within data and probabilistic classification of data according to model mixtures;

initiating a learning algorithm, by the computer processor, causing the computer processor to execute the computer readable and executable program for simultaneously discovering mixtures of models within data and probabilistic classification of data according to mixture models of a plurality of models;

applying a random sampling operation to determine mathematical functions;

determining multiple models of the plurality of models that fit portions of mixture models of the plurality of models;

probabilistically assigning points to multiple models of the plurality of models by

using abstractions of mathematical functions to form simulated equivalent mathematical functions, causing one or more mathematical functions to be processed as one or more of the plurality of models;

comparing different mathematical functions, using geometric properties, including overlap, supporting point sets, and density; and

providing user settable thresholds for user interaction with computations of residual error and corresponding and supporting point sets to learned mixture models.

2. The method according to claim 1, wherein probabilistically assigning points to multiple models of the plurality of models includes generating a confidence rating that each point of the cluster of points in the first region of the domain corresponds to the first learned mixture model and generating a confidence rating that each point of the cluster of points in the first region of the domain correspond to the second learned mixture model and causing determination of a behavior of a system described by the learned mixture models.

3. The method according to claim 1, wherein the random sampling operation includes one of PROGRESSIVE SAMPLE CONSENSUS (PROSAC), RANDOM SAMPLE CONSENSUS (RANSAC), and MONTE CARLO operations.

4. The method according to claim 3, wherein applying the random sampling operation to determine mathematical functions determines mathematical functions consistent with a dataset of a cluster of points in a first region of a domain and a set of points distributed near one of a first line and a mathematical function, in the first region of the domain.

5. The method according to claim 4, wherein the mathematical function includes one or more of a transcendental hyperbolic function and a polynomial function.

6. A system of discovering mixtures of models within data and probabilistic classification of data according to model mixtures, the system comprising:

a computer processor having a display, an input device and an output device;

a network interface communicatively coupling the computer processor to a network; and

a memory having a dynamic repository, an algorithm unit and a program unit containing a computer readable and computer executable program; and

a memory controller communicatively coupling the computer processor with contents of the dynamic repository, the algorithm unit and the computer readable and computer executable program residing in the program unit, wherein when executed by the computer processor, the computer readable and computer executable program causes the computer processor to perform operations of discovering mixtures of models including operations of;

receiving, a request for discovering mixtures of models within data and probabilistic classification of data according to model mixtures;

initiating a learning algorithm, by the computer processor, causing the computer processor to execute the computer readable and executable program discovering mixtures of models within data and probabilistic classification of data according to mixture models;

applying a random sampling operation to determine mathematical functions;

determining, by the computer processor, one of when a data set consists of a cluster of points in a first region of a domain, determining when a set of points distributed near a first line that extends across part of the domain exists;

inferencing parameters, of the first line, that one of describe the set of points distributed near the first line, and describe a mean and variance of the cluster of points in the first region of the domain creating a description of the cluster of points in the first region of the domain, and describe other parameters needed to describe an instance of a function in a number of dimensions;

computing, by the computer processor, approximations of a first learned mixture model corresponding to the set of points distributed near the first function and a second learned mixture model corresponding to the set of points near the second function within the domain and similar approximations for functions determined to exist within data;

probabilistically assigning points to multiple models of the plurality of models;

using abstractions of mathematical functions to form simulated equivalent mathematical functions, causing one or more mathematical functions to be processed as one or more of the plurality of models;

comparing different mathematical functions, using geometric properties, including overlap, supporting point sets, and density;

providing user settable thresholds, including support and overlap, for user interaction with computations of the number of and associated residual error and corresponding to the first and second learned mixture models; and

generating a confidence rating that each point of the cluster of points in the first region of the domain corresponds to the first learned mixture model and generating a confidence rating that each point of the cluster of points in the first region of the domain correspond to the second learned mixture model and causing determination of a behavior of a system described by the learned mixture models.

7. The system according to claim 6, having the computer processor performing operations of discovering mixtures by performing operations of determining, wherein determining, by the computer processor, further includes determining when a set of points constitutes one of a transcendental, hyperbolic, polynomial, and other function, which is described as embedded in any number of dimensions that describe input data.

8. The system according to claim 6, having the computer processor performing operations of discovering mixtures by performing operations of probabilistically assigning points to multiple models of the plurality of models, wherein probabilistically assigning points, by the computer processor, further includes:

determining a first probability that the first learned mixture model corresponds to each point of the cluster of points in the first region of the domain and determining a second probability that the second learned mixture model corresponds to each point of the set of points distributed near the first line, wherein determining the first and second probabilities is performed by testing each point, wherein determining the first and second probabilities eliminates a requirement for a fit of each point displaced from a true position, wherein setting a minimum number of points for each of the first and second learned mixture models distinguishes the first and second learned mixture models from a combination learned mixture model formed from parameters of the first and second learned mixture models, wherein determining the first and second probabilities includes assigning a fixed percent probability up to about fifty percent for points of a line, depending on a residual error fit of the first and second learned mixture model, and wherein the learning algorithm probabilistically determines whether a series of Gaussian mixture models are found, by combining a number of points of the first and second learned mixture models with an average residual points to be excluded and repeating probabilistically assigning points to multiple models of the plurality of models for each function determined to exist within data.

9. The system according to claim 6, having the computer processor applying a random sampling operation to determine mathematical functions, wherein the random sampling operation includes one of PROGRESSIVE SAMPLE CONSENSUS (PROSAC), RANDOM SAMPLE CONSENSUS (RANSAC), and MONTE CARLO operations.

10. The system according to claim 9, having the computer processor applying the random sampling operation, determining mathematical functions, further causes determining mathematical functions consistent with a dataset of a cluster of points in a first region of a domain and a set of points distributed near one of a first line and a mathematical function, in the first region of the domain.

11. The system according to claim 10, wherein the mathematical function includes one or more of a transcendental hyperbolic function and a polynomial function.

12. A computer readable medium having a plurality of computer executable instructions in the form of a computer readable and computer executable program executed by a computer processor causing the computer processor to perform a method of discovering mixtures of models within data and probabilistic classification of data according to model mixtures, the plurality of computer executable instructions including:

instructions causing receiving, a request for discovering mixtures of models within data and probabilistic classification of data according to model mixtures;

instructions initiating a learning algorithm, by the computer processor;

instructions for applying a random sampling operation to determine mathematical functions;

instructions causing determining, by the computer processor, one of when a data set consists of a cluster of points in a first region of a domain, determining when a set of points distributed near a first line that extends across part of the domain exists;

instructions causing inferencing parameters, of the first line, that one of describe the set of points distributed near the first line, and describe a mean and variance of the cluster of points in the first region of the domain creating a description of the cluster of points in the first region of the domain, and describe other parameters needed to describe an instance of a function in a plurality of dimensions;

instructions causing computing, by the computer processor, approximations of a first learned mixture model corresponding to the set of points distributed near a first function and a second learned mixture model corresponding to the set of points near a second function within the domain and similar approximations for functions determined to exist within data embedded in any subspace of the domain and total domain;

instructions causing probabilistically assigning points to multiple models of the plurality of models;

instructions for using abstractions of mathematical functions to form simulated equivalent mathematical functions, causing one or more mathematical functions to be processed as one or more of the plurality of models;

instructions causing comparing different mathematical functions, using geometric properties, including overlap, supporting point sets, and density;

instructions for providing a user settable threshold for user interaction with computations of the number of and associated residual error and corresponding to the first and second learned mixture models; and

instructions for generating a confidence rating that each point of the cluster of points in the first region of the domain corresponds to the first learned mixture model and generating a confidence rating that each point of the cluster of points in the first region of the domain correspond to the second learned mixture model and causing determination of a behavior of a system described by the learned mixture models.

13. The instructions of the computer readable medium according to claim 12, initiating a learning algorithm, by the computer processor, further include instructions causing the computer processor to execute the computer readable and executable program discovering mixtures of models within data and probabilistic classification of data according to mixture models.

14. The instructions of the computer readable medium according to claim 12, causing determining, by the computer processor, one of when a data set consists of a cluster of points in a first region of a domain, determining when a set of points distributed near a first line that extends across part of the domain exists, includes further instructions causing determining when a set of points constitutes one of a transcendental, hyperbolic, polynomial, and other function, which is described as embedded in any number of dimensions that describe input data.

15. The instructions of the computer readable medium according to claim 12, causing instructions for probabilistically assigning points to multiple models of the plurality of models, further includes instructions causing:

determining a first probability that the first learned mixture model corresponds to each point of the cluster of points in the first region of the domain and determining a second probability that the second learned mixture model corresponds to each point of the set of points distributed near the first line, wherein determining the first and second probabilities is performed by testing each point, wherein determining the first and second probabilities eliminates a requirement for a fit of each point displaced from a true position, wherein setting a minimum number of points for each of the first and second learned mixture models distinguishes the first and second learned mixture models from a combination learned mixture model formed from parameters of the first and second learned mixture models, wherein determining the first and second probabilities includes assigning a fixed percent probability up to about fifty percent for points of a line, depending on a residual error fit of the first and second learned mixture model, and wherein the learning algorithm probabilistically determines whether a series of Gaussian mixture models are found, by combining a number of points of the first and second learned mixture models with an average residual points to be excluded and repeating probabilistically assigning points to multiple models of the plurality of models for each function determined to exist within data.

16. The instructions of the computer readable medium according to claim 12, of applying a random sampling operation to determine mathematical functions, further includes instructions calling an algorithm to perform one of PROGRESSIVE SAMPLE CONSENSUS (PROSAC), RANDOM SAMPLE CONSENSUS (RANSAC), and MONTE CARLO operations.

17. The instructions of the computer readable medium according to claim 16, wherein applying the random sampling operation to determine mathematical functions determines mathematical functions consistent with a dataset of a cluster of points in a first region of a domain and a set of points distributed near one of a first line and a mathematical function, in the first region of the domain.

18. The instructions of the computer readable medium according to claim 17, wherein the mathematical function includes one or more of a transcendental hyperbolic function and a polynomial function.