Machine Learning
Artificial Intelligence - Design and analysis of intelligent agents
Machine learning deals with construction and study of systems that can learn from data, rather that follow explicitly programmed instructions.
Every machine learning algorithm has three components:
- Representation - How would you characterise what is being learnt
- Evaluation - How would you like to measure the goodness of what is being learnt
- Optimization - Given the evaluation and characterization, find the optimum representation.
Types of Learning
- Supervised Learning - Given a set of labelled training examples, learn a function. (Classification, Regression)
- Unsupervised Learning - Given a set of unlabelled training examples, learn a function that identifies similar groups (Clustering, Collaborative Filtering, Dimensional Reduction)
- Semi-supervised Learning - Minimal supervision
- Transfer Learning - Transfer knowledge between domains
- Active Learning - Learning agent actively queries a source to label new tricky data points.
- Online Learning - Learning on the go
- Reinforcement Learning - Learning based on rewards from environment
Data Split
- Binary Split - Testing and Training (Overfitting, too late to repair)
- Ternary Split - Train, Validation, Test To tackle issues with binary split, early stopping may be adopted.
Small Datasets
| Method | Best For | Computational Cost | Main Benefit |
|---|---|---|---|
| Simple Split | Large datasets | Low | Fast; good for quick prototyping. |
| K-Fold | Medium/Small data | Moderate ($K \times$ training) | Robust; uses all data for validation. |
| LOOCV | Very small data | High ($N \times$ training) | Unbiased; maximum data usage for training. |