Classification can be divided into supervised and unsupervised classification. In supervised classification, also termed discrimination, a set of data samples consisting of a set of variables is available. All the samples in the data set are labelled; they are thus all assigned to a specific class. With unsupervised classification, sometimes termed clustering, the samples in the data set are not labelled.

Pattern - A pattern is a  d-dimensional vector x =  (x1,...,xd) where each element of the vector corresponds to the value of a different feature. Features are used to describe an observation. A pattern is also referred to as a sample or observation.

Feature extraction - The process of extracting certain attributes from collected data. These attributes are used to map the original data to a feature space in which the data will be seperable so that classification can be performed.

Feature selection - The process of selecting the most relevant features that have been extracted. Thus selecting the features that will map to the optimal feature space to perform classification.

Generalization describes how well an algorithm will perform on functions/data it has not yet seen, given that the unseen functions/data are drawn from the same sample distribution as the training data.