Class weights fix imbalance without touching the data at all — instead, the model's cost function is adjusted to penalize mistakes on the minority class more heavily than mistakes on the majority class.
Formula — The Standard "Balanced" Weighting
\(n\) is the total number of samples, \(k\) is the number of classes, and \(n_c\) is the number of samples in class \(c\). Rarer classes get a larger weight — their errors count for more in the loss function.
Worked Example
Reference dataset: \(n=1000\), \(k=2\) classes, \(n_0=950\) (majority), \(n_1=50\) (minority).
The minority class gets a weight roughly 19 times larger than the majority class (\(10.0/0.526\approx 19\)) — every minority-class mistake during training now counts as if it were about 19 separate majority-class mistakes, forcing the model to pay real attention to getting it right.
from sklearn.utils.class_weight import compute_class_weight
import numpy as np
y_train = np.array([0]*950 + [1]*50)
weights = compute_class_weight(class_weight="balanced", classes=np.array([0,1]), y=y_train)
print(weights) # [0.526, 10.0] -- matches the hand calculation
from sklearn.linear_model import LogisticRegression
model = LogisticRegression(class_weight="balanced") # scikit-learn computes this automatically
model.fit(X_train, y_train)Custom Weights — When "Balanced" Isn't Quite Right
# If missing fraud costs 10x more than a false alarm (a business-specific ratio,
# not necessarily the exact inverse-frequency "balanced" weighting)
model = LogisticRegression(class_weight={0: 1, 1: 10})
model.fit(X_train, y_train)The "balanced" formula is a reasonable default, but the actual right ratio should reflect the real business cost of each error type when that information is available — sometimes that happens to match inverse class frequency, sometimes it doesn't.
Class Weights vs Resampling — The Key Advantage
| Resampling (SMOTE/Over/Undersampling) | Class Weights | |
|---|---|---|
| Changes the data? | Yes | No — original data untouched |
| Extra computation/memory | Yes — larger or synthetic dataset | No — same dataset size |
| Works with any model? | Yes, since it's a data-level fix | Only models that support a class-weight parameter |
Class weights are often the simplest first thing to try — a single parameter change, no data pipeline modifications, no risk of the duplication or synthetic-point issues resampling techniques carry.
Practical Use Cases
- Any classifier that supports a
class_weightparameter (logistic regression, SVM, decision trees, Random Forest) — a low-effort first fix to try - Situations where the real per-error business cost is known and doesn't match simple inverse frequency
Common Mistakes
- Assuming
class_weight="balanced"is always the theoretically "correct" weighting — it's a reasonable default based on inverse frequency, not necessarily the ratio that matches real business costs. - Not checking whether a specific algorithm even supports class weights before assuming this technique is available.
Interview Relevance
Q: "Why might you prefer class weights over SMOTE for handling imbalance?" Class weights require no changes to the underlying data — no duplication risk, no synthetic points, no increased dataset size or training time — making it the simplest and often the first technique worth trying, especially when the algorithm being used directly supports a class-weight parameter.
Practice Question
For a dataset with 3 classes of sizes 800, 150, 50 (total 1000), compute the "balanced" class weight for each class using the formula above.
