Skip to content

Basics

Why is logistic repression considered a linear model?

Logistic regression is considered a linear model because the output always depends on the sum of inputs and parameters rather than the product.

  • Produces a linear decision surface
  • The net input function is simply the dot product of our input features and respective model coefficients

Class Imbalance

A dataset where the distribution of labels is highly skewed, with one majority class significantly outnumbering the minority class. Can affect the accuracy as the model can simply predict the majority class and get high accuracy.

Techniques to resolve class imbalance

Data-Level Techniques

1) Resampling - Random Under-sampling / Random over-sampling / SMOTE

Under-sampling - loss of information Over-sampling - risk of overfitting 2) SMOTE (Synthetic Minority Over-sampling techniques) - Instead of duplicating data, SMOTE selects a minority point and its nearest neighbours and creates new synthetic points along the lines connecting them. 3) Balanced Mini-batch Training - Instead of balancing the entire dataset at once, you force every batch (32 samples) to have a 50/50 distribution.

Algorithm-Level Techniques

Instead of changing the data, we can change the algorithm

1) Cost-Sensitive Learning (Class Weighing) - You assign a higher cost to misclassifying a minority example than a majority one. 2) Change the Performance Metric - Use recall, Precision, F1-Score, AUC-ROC

Performance Metrics

  • Precision (Quality) - TP/TP+FP (Of all the times the model said it was a disaster, how many times it was actually right)
  • Recall (Sensitivity) - TP/TP+FN (Of all the actual disasters, how many did the model manage to catch)
  • F1-Score (Balanced Metric) - The harmonic mean of the two.
  • ROC (Receiver Operating Characteristic) - A plot showing how the True Positive compares to False Positive as you move that threshold from 0% to 100% Pasted image 20260419020359.png
  • AUC(Area Under the Curve) Pasted image 20260419020555.png