The highest tagged major version is
Documentation
¶
Overview ¶
Package classification is a require()-able host module exposing supervised classifiers to .lsc code: a Naive Bayesian text classifier, k-nearest-neighbours, a perceptron, logistic regression, and a kernel SVM.
Each constructor returns an object (a table whose methods close over the underlying Go model), so usage reads as `clf:learn(...)`, `clf:fit(...)`, `clf:predict(...)`, etc.
Index ¶
- Variables
- func RegisterClassificationPreload(v *vm.VM)
- type Class
- type Classifier
- func (c *Classifier) AddClass(class Class) error
- func (c *Classifier) Classify(document []string) (Class, []float64, bool)
- func (c *Classifier) ClassifyProb(document []string) (Class, []float64, bool)
- func (c *Classifier) ClassifySafe(document []string) (Class, []float64, bool, error)
- func (c *Classifier) ConvertTermsFreqToTfIdf()
- func (c *Classifier) IsTfIdf() bool
- func (c *Classifier) Learn(document []string, which Class)
- func (c *Classifier) Learned() int
- func (c *Classifier) LogScores(document []string) ([]float64, int, bool)
- func (c *Classifier) Observe(word string, count int, which Class)
- func (c *Classifier) ProbScores(doc []string) ([]float64, int, bool)
- func (c *Classifier) ReadClassFromFile(class Class, location string) (err error)
- func (c *Classifier) SafeProbScores(doc []string) ([]float64, int, bool, error)
- func (c *Classifier) Seen() int
- func (c *Classifier) WordCount() []int
- func (c *Classifier) WordFrequencies(words []string) [][]float64
- func (c *Classifier) WordsByClass(class Class) (freqMap map[string]float64)
- func (c *Classifier) WriteClassToFile(name Class, rootPath string) error
- func (c *Classifier) WriteClassesToFile(rootPath string) error
- func (c *Classifier) WriteGob(w io.Writer) (err error)
- func (c *Classifier) WriteToFile(name string) error
- type KNN
- type LogisticRegression
- type Perceptron
- type SVM
- type SVMConfig
- type SVMKernel
Constants ¶
This section is empty.
Variables ¶
var ErrAlreadyConverted = errors.New("cannot add class after TF-IDF conversion")
var ErrClassExists = errors.New("class already exists")
var ErrUnderflow = errors.New("possible underflow detected")
Functions ¶
func RegisterClassificationPreload ¶
RegisterClassificationPreload installs the loader under package.preload; the module table is built lazily on the first require("classification").
Types ¶
type Classifier ¶
type Classifier struct {
Classes []Class
DidConvertTfIdf bool // we cannot classify a TF-IDF classifier if we haven't yet called ConvertTermsFreqToTfIdf
// contains filtered or unexported fields
}
Classifier implements the Naive Bayesian Classifier.
func NewClassifier ¶
func NewClassifier(classes ...Class) *Classifier
NewClassifier returns a new classifier. The classes provided should be at least 2 in number and unique, or this method will panic.
func NewClassifierFromFile ¶
func NewClassifierFromFile(name string) (*Classifier, error)
NewClassifierFromFile loads an existing classifier from file. The classifier was previously saved with a call to c.WriteToFile(string).
func NewClassifierFromReader ¶
func NewClassifierFromReader(r io.Reader) (*Classifier, error)
NewClassifierFromReader: This actually does the deserializing of a Gob encoded classifier
func NewClassifierTfIdf ¶
func NewClassifierTfIdf(classes ...Class) *Classifier
NewClassifierTfIdf returns a new TF-IDF classifier. The classes provided should be at least 2 in number and unique, or this method will panic.
func (*Classifier) AddClass ¶
func (c *Classifier) AddClass(class Class) error
AddClass adds a new class to the classifier dynamically. Returns ErrClassExists if the class already exists, or ErrAlreadyConverted if the classifier has been converted to TF-IDF. This method is safe for concurrent use.
func (*Classifier) Classify ¶
func (c *Classifier) Classify(document []string) (Class, []float64, bool)
Classify returns the most likely class for the given document along with the log scores and whether the classification is strict. This is a convenience wrapper around LogScores that returns the Class directly instead of an index.
func (*Classifier) ClassifyProb ¶
func (c *Classifier) ClassifyProb(document []string) (Class, []float64, bool)
ClassifyProb returns the most likely class for the given document along with the probability scores and whether the classification is strict. This is a convenience wrapper around ProbScores that returns the Class directly instead of an index.
func (*Classifier) ClassifySafe ¶
ClassifySafe returns the most likely class for the given document along with the probability scores, whether the classification is strict, and an error if underflow is detected. This is a convenience wrapper around SafeProbScores that returns the Class directly instead of an index.
func (*Classifier) ConvertTermsFreqToTfIdf ¶
func (c *Classifier) ConvertTermsFreqToTfIdf()
ConvertTermsFreqToTfIdf uses all the TF samples for the class and converts them to TF-IDF https://en.wikipedia.org/wiki/Tf%E2%80%93idf once we have finished learning all the classes and have the totals. This method is safe for concurrent use.
func (*Classifier) IsTfIdf ¶
func (c *Classifier) IsTfIdf() bool
IsTfIdf returns true if we are a classifier of type TfIdf
func (*Classifier) Learn ¶
func (c *Classifier) Learn(document []string, which Class)
Learn will accept new training documents for supervised learning. This method is safe for concurrent use.
func (*Classifier) Learned ¶
func (c *Classifier) Learned() int
Learned returns the number of documents ever learned in the lifetime of this classifier. This method is safe for concurrent use.
func (*Classifier) LogScores ¶
func (c *Classifier) LogScores(document []string) ([]float64, int, bool)
LogScores produces "log-likelihood"-like scores that can be used to classify documents into classes.
The value of the score is proportional to the likelihood, as determined by the classifier, that the given document belongs to the given class. This is true even when scores returned are negative, which they will be (since we are taking logs of probabilities).
The index j of the score corresponds to the class given by c.Classes[j].
Additionally returned are "inx" and "strict" values. The inx corresponds to the maximum score in the array. If more than one of the scores holds the maximum values, then strict is false.
Unlike c.Probabilities(), this function is not prone to floating point underflow and is relatively safe to use. This method is safe for concurrent use.
func (*Classifier) Observe ¶
func (c *Classifier) Observe(word string, count int, which Class)
Observe should be used when word-frequencies have been already been learned externally (e.g., hadoop). This method is safe for concurrent use.
func (*Classifier) ProbScores ¶
func (c *Classifier) ProbScores(doc []string) ([]float64, int, bool)
ProbScores works the same as LogScores, but delivers actual probabilities as discussed above. Note that float64 underflow is possible if the word list contains too many words that have probabilities very close to 0.
Notes on underflow: underflow is going to occur when you're trying to assess large numbers of words that you have never seen before. Depending on the application, this may or may not be a concern. Consider using SafeProbScores() instead.
If all scores underflow to zero, returns equal probabilities for all classes (1/n each). This method is safe for concurrent use.
func (*Classifier) ReadClassFromFile ¶
func (c *Classifier) ReadClassFromFile(class Class, location string) (err error)
ReadClassFromFile loads existing class data from a file. This method is safe for concurrent use.
func (*Classifier) SafeProbScores ¶
SafeProbScores works the same as ProbScores, but is able to detect underflow in those cases where underflow results in the reverse classification. If an underflow is detected, this method returns an ErrUnderflow, allowing the user to deal with it as necessary. Note that underflow, under certain rare circumstances, may still result in incorrect probabilities being returned, but this method guarantees that all error-less invocations are properly classified.
Underflow detection is more costly because it also has to make additional log score calculations.
When underflow is detected, the returned scores are computed from log-domain scores using the log-sum-exp trick for numerical stability. This method is safe for concurrent use.
func (*Classifier) Seen ¶
func (c *Classifier) Seen() int
Seen returns the number of documents ever classified in the lifetime of this classifier.
func (*Classifier) WordCount ¶
func (c *Classifier) WordCount() []int
WordCount returns the number of words counted for each class in the lifetime of the classifier. This method is safe for concurrent use.
func (*Classifier) WordFrequencies ¶
func (c *Classifier) WordFrequencies(words []string) [][]float64
WordFrequencies returns a matrix of word frequencies that currently exist in the classifier for each class state for the given input words. In other words, if you obtain the frequencies
freqs := c.WordFrequencies(/* [j]string */)
then the expression freq[i][j] represents the frequency of the j-th word within the i-th class. This method is safe for concurrent use.
func (*Classifier) WordsByClass ¶
func (c *Classifier) WordsByClass(class Class) (freqMap map[string]float64)
WordsByClass returns a map of words and their probability of appearing in the given class. This method is safe for concurrent use.
func (*Classifier) WriteClassToFile ¶
func (c *Classifier) WriteClassToFile(name Class, rootPath string) error
WriteClassToFile writes a single class to file. This method is safe for concurrent use.
func (*Classifier) WriteClassesToFile ¶
func (c *Classifier) WriteClassesToFile(rootPath string) error
WriteClassesToFile writes all classes to files. This method is safe for concurrent use.
func (*Classifier) WriteGob ¶
func (c *Classifier) WriteGob(w io.Writer) (err error)
WriteGob serializes this classifier to GOB and writes to Writer. This method is safe for concurrent use.
func (*Classifier) WriteToFile ¶
func (c *Classifier) WriteToFile(name string) error
WriteToFile serializes this classifier to a file. This method is safe for concurrent use.
type KNN ¶
type KNN struct {
// contains filtered or unexported fields
}
KNN is a lazy k-nearest-neighbours classifier. It stores the training set verbatim and, at predict time, votes among the k closest samples by Euclidean distance. Works on numeric feature vectors with arbitrary string labels.
func NewKNN ¶
NewKNN returns a KNN classifier that votes over the k nearest samples. k is clamped to at least 1.
type LogisticRegression ¶
type LogisticRegression struct {
// contains filtered or unexported fields
}
LogisticRegression is a binary classifier trained by batch gradient descent on the log-loss. Unlike the perceptron it also yields calibrated class probabilities via the logistic (sigmoid) link.
func NewLogisticRegression ¶
func NewLogisticRegression(lr float64, epochs int) *LogisticRegression
NewLogisticRegression builds a logistic-regression model with the given learning rate and epoch budget, applying defaults for non-positive values.
func (*LogisticRegression) Fit ¶
func (m *LogisticRegression) Fit(features [][]float64, labels []string)
Fit trains the model with full-batch gradient descent.
func (*LogisticRegression) Predict ¶
func (m *LogisticRegression) Predict(x []float64) string
Predict returns the predicted class label for x.
func (*LogisticRegression) PredictProba ¶
func (m *LogisticRegression) PredictProba(x []float64) float64
PredictProba returns P(positive class | x) in [0, 1].
type Perceptron ¶
type Perceptron struct {
// contains filtered or unexported fields
}
Perceptron is the classic online linear classifier. It performs the perceptron update (w += lr·(y−ŷ)·x) over the data for a number of epochs. Converges to a separating hyperplane when the data is linearly separable; otherwise it just stops after the epoch budget.
func NewPerceptron ¶
func NewPerceptron(lr float64, epochs int) *Perceptron
NewPerceptron builds a perceptron with the given learning rate and epoch budget. Sensible defaults are applied for non-positive values.
func (*Perceptron) Fit ¶
func (p *Perceptron) Fit(features [][]float64, labels []string)
Fit trains the perceptron on the labelled feature rows.
func (*Perceptron) Predict ¶
func (p *Perceptron) Predict(x []float64) string
Predict returns the predicted class label for x.
type SVM ¶
type SVM struct {
// contains filtered or unexported fields
}
SVM is a binary classifier. After Fit it retains only the support vectors (training points with a non-zero Lagrange multiplier), which are all that the decision function needs.
func NewSVM ¶
NewSVM builds an untrained SVM from the given configuration, applying defaults for any unset field.
func (*SVM) DecisionFunction ¶
DecisionFunction exposes the signed margin: positive favours the positive class, and the magnitude reflects distance from the boundary.
func (*SVM) Fit ¶
Fit trains the SVM on labelled feature rows using simplified SMO. The first label seen becomes the negative (−1) class, the second positive (+1).
func (*SVM) SupportVectorCount ¶
SupportVectorCount reports how many training points became support vectors.
type SVMConfig ¶
type SVMConfig struct {
Kernel SVMKernel
Gamma float64
Coef0 float64
Degree int
C float64
Tol float64
MaxIter int
Seed int64
}
SVMConfig carries the hyper-parameters for NewSVM. Zero fields fall back to sensible defaults.
type SVMKernel ¶
type SVMKernel int
SVMKernel identifies the kernel used to map inputs into feature space.
func ParseKernel ¶
ParseKernel maps a string name to an SVMKernel, defaulting to RBF.
Source Files