Classification predicts a category from labelled data. Build decision trees by the purest split, classify with k nearest neighbours, reason with Bayes, and measure it all honestly — then test yourself with 10 questions.
Given training records with a known class label, a classifier learns a model that maps attributes → class. New, unlabelled records are then assigned a class. It's supervised learning because the labels guide training.
General approach: split into train/test → build the model on training data → predict on the test set → evaluate accuracy → deploy.
Summarise the general traits of a target class.
The surface a model draws to separate classes.
A model that memorises training noise and fails to generalise.
Combine many models (bagging, boosting, random forest) for accuracy.
A decision tree splits the data with attribute tests, choosing at each node the split that makes the children purest. Purity is measured by Gini or entropy; information gain is the drop in entropy after a split. Slide the class mix and watch impurity respond — purest (0) at the ends, most impure (max) at a 50/50 mix.
Binary, multiway, or threshold splits on nominal, ordinal or continuous attributes.
Gini index, entropy/information gain, gain ratio — pick the attribute that purifies most.
Recursively split nodes until pure or a stop condition; then prune to avoid overfitting.
KNN is a lazy learner — it stores the training data and classifies a new point by a majority vote of its k closest neighbours. Click anywhere to drop a query point; lines connect it to its k neighbours and it takes the majority colour. Slide k and watch the verdict change.
Applies Bayes' theorem assuming features are conditionally independent given the class. Fast, surprisingly accurate, needs little data.
P(class∣x) ∝ P(x∣class)·P(class). Pick the class with the highest posterior probability.
A graph of variables with conditional dependencies — relaxes the naïve independence assumption to model real relationships.
KNN is lazy (no model until query time); trees and Bayes are eager (build a model up front).
Accuracy, error rate, precision, recall, F-measure; cross-validation gives a reliable estimate.
Bagging (e.g. random forest), boosting (e.g. AdaBoost) and stacking combine weak models into a strong one.
Clear explainers from StatQuest with Josh Starmer for the three classifiers in this unit.
Videos are embedded from YouTube and belong to their creators. If a frame is blank, your network may block YouTube.