Multi-Class Market Regime Detection

Detecting market regimes (bull, bear, sideways) helps adapt trading strategies. LightGBM's multi-class classification makes this straightforward.

Defining Regimes

def classify_regime(prices, lookback=20, threshold=0.02):
    """Classify market regime."""
    returns = prices.pct_change(lookback)
    
    regimes = pd.Series(index=prices.index, dtype=int)
    regimes[returns > threshold] = 0  # BULL
    regimes[returns < -threshold] = 1  # BEAR
    regimes[(returns >= -threshold) & (returns <= threshold)] = 2  # SIDEWAYS
    
    return regimes

LightGBM Multi-Class

import lightgbm as lgb

# Prepare data
X = create_regime_features(nifty_data)
y = classify_regime(nifty_data['Close'])

# Multi-class parameters
params = {
    'objective': 'multiclass',
    'num_class': 3,
    'metric': 'multi_logloss',
    'boosting_type': 'gbdt',
    'num_leaves': 15,
    'max_depth': 4,
    'learning_rate': 0.05,
    'feature_fraction': 0.8,
    'min_child_samples': 30
}

# Train
train_data = lgb.Dataset(X_train, label=y_train)
model = lgb.train(params, train_data, num_boost_round=500)

# Predict (returns probabilities for each class)
regime_probs = model.predict(X_test)  # Shape: (n_samples, 3)
regime_pred = regime_probs.argmax(axis=1)  # Most likely regime

Walk-Forward Results

Regime detection on Nifty 50 (2020-2025):

  • Bull detection: 73% accuracy
  • Bear detection: 69% accuracy
  • Sideways detection: 62% accuracy
  • Overall accuracy: 68%

Regime-Adaptive Strategy

def regime_strategy(regime, direction_signal):
    """Adapt strategy based on regime."""
    if regime == 0:  # BULL
        return direction_signal  # Follow trend
    elif regime == 1:  # BEAR
        return -direction_signal  # Reverse or hedge
    else:  # SIDEWAYS
        return 0  # No position

Applications

  • Position sizing: Larger in trends, smaller in sideways
  • Strategy selection: Momentum in bull, mean-reversion in sideways
  • Risk management: Tighter stops in bear markets

Labeling Regimes: Rules vs Hidden Markov Models

Multi-class regime models live or die on the label definition. Two approaches dominate, and they disagree by design:

  • Rule-based labels: rolling 20-day return above +4% is bull, below -4% is bear, else sideways; simple, reproducible, and good enough to deploy.
  • HMM labels: a Gaussian hidden Markov model infers latent states from returns and volatility; more data-driven but sensitive to state count and refits.

For a LightGBM that must be explainable to a trader, rule-based labels audited against an HMM as a cross-check strike the best balance. Label with the rule so the model learns a defensible frontier, then let the HMM dispute rare boundary cases.

Calibrating Multi-Class Probabilities

A three-class output loses meaning if probabilities are uncalibrated. LightGBM's raw logits are not probabilities:

  1. Use the built-in multiclass objective and apply a softmax, then calibrate with temperature scaling or isotonic regression on a validation fold.
  2. Validate calibration per class with reliability curves: when the model says 70% sideways, does sideways actually happen about 70% of the time?
  3. Watch the classifier's edge cases: the worst model mistake is a confident bull call on a creeping decline, and calibration exposes it.

For regime products this matters more than raw accuracy, because a strategy triggered off "prob(bull) > 0.6" does not care about accuracy, it cares about the threshold being honoured.

The Transition Matrix as a Decision Tool

Rather than reading today's regime, traders should read the transition matrix the model implies:

  • Rows are today's regime, columns tomorrow's; the diagonal dominance tells you persistence.
  • If P(sideways stays sideways) is 0.82, a sideways signal supports a 10-day theta trade.
  • If P(bull to bear) climbs above 0.2, any long-premium position should carry a stop.

Published in the morning brief, the matrix converts a classification output into an action calendar: decay a stay, hedge a switch.

Second-Stage Regime Filters

Pure regime models produce muddy signals right at transitions. A second stage cleans them:

  • Require a regime probability above 0.55 and a two-day confirmation before trading it.
  • Disable new entries in the no-man's zone where the top two probabilities sit within 0.05 of each other.
  • Apply a regime-specific overlay: leverage only in confirmed bull, insurance (long puts) only as bear gains probability.

Walk-Forward Scheduling That Matches Regimes

Retraining cadence should track the regime horizon, not a fixed date. A workable schedule for Nifty daily data:

  1. Retrain every Monday using a rolling 3-year window.
  2. Keep a monthly regime-recall validation; if the model's regime forecast diverges from the rule-based label for more than a week, freeze new signals until audit.
  3. Store every retrained checkpoint with its evaluation, so a bad regress is reversible in minutes, not weeks.

The regime may not be predictable, but your response to it must be; multi-class LightGBM is at its best when it is a slow, clearly-communicated map instead of a fast, silent oracle.

Regime Classification Design

Keep the number of regimes between three and five, label them from volatility and trend characteristics, and evaluate with a confusion matrix rather than accuracy alone. Train class weights to counter imbalance, and never predict a regime switch on the closing bar without volume confirmation. A regime model is a context filter; pair its output with a regime-appropriate strategy rather than trading the label directly.