Market Microstructure
Study of how exchanges operate, including order types, matching, and price formation.
Order Flow Features
- Bid-ask spread dynamics
- Order imbalance
- Trade size distribution
- Time between trades
ML Applications
- Price impact estimation
- Optimal execution
- Market making algorithms
- Liquidity prediction
Data Requirements
Tick-level data required. Complete order book snapshots. High-frequency data providers needed.
What Order-Flow Data Adds That Price Bars Cannot
Daily candles hide the most valuable information in the market: the sequence and aggression of individual orders. Order-flow analysis reconstructs whether current price is being advanced by passive limit orders, aggressive market orders, or a mix of both. Machine-learning models trained on flow features routinely detect turning points earlier than models that only see close prices, simply because the flow begins to weaken before the price pattern completes.
Key Flow Features to Engineer
- Trade intensity: number of prints per minute and their average size.
- Bid-ask imbalance: difference between resting bid volume and resting ask volume at the touch.
- Order book depth: cumulative size within the first five levels on each side.
- Sweep size: trade size exceeding the displayed depth, indicating aggressive orders.
- Transaction cost: spread paid above the mid-price, proxying pressure.
Label Building From Trades
Supervised flow models need a label, and naive labels like 'price up in the next minute' encode noise. A more robust label is the drift at the 5-minute horizon minus the transaction cost: a trade is only 'good' if the subsequent move repays the spread and the fee. Binary labels built from the realised mid-price after three minutes reduce the clutter and teach the model to predict actionable drift rather than random flicker.
Model Choice for Flow Data
Order-book features are highly correlated, so logistic regression on carefully engineered imbalance features is a strong baseline and surprisingly hard to beat. When you want more capacity, gradient boosting handles the non-linear interactions between depth and intensity; deep learning pays off mainly when you train on the raw L2 book sequence, which requires serious data engineering to feed as tensors.
A Practical Pipeline
- Capture every trade and book snapshot for the contract of interest for 60 trading days.
- Build minute-level features: trade count, volume, imbalance, depth, sweep frequency.
- Create 3-minute-forward labels adjusted for spread cost.
- Train a gradient-boosted classifier with early stopping on a later time window.
- Test on the last two weeks; expect modest but persistent accuracy above 0.53.
Where the Edge Actually Lives
The repeatable edge in flow data is short-lived: it decays within minutes and disappears entirely in the last hour of the expiry day as volume thins. Legitimate applications are execution timing (choosing the next 30 seconds to send a large order) and early alerting on index reversals, not turning every flow print into a trade. Treat flow-ML output as a timing filter layered on top of a broader strategy, and you avoid the trap of overtrading noise.
Tick Data Hygiene Before Any Model
Order flow analysis lives or dies on the quality of the tick file. The first cleanup job is deduplication: Indian exchange consolidation and broker feeds both retransmit ticks, and a double-counted quote will poison any feature built on trade size. Stamp every record with the exchange timestamp, not the receiving machine's clock, because latency variance between your feed and the exchange's bus destroys the ordering that the whole field depends on. Then strip auction prints, odd-lot entries, and block-deal markers from the flow or train a model that believes an off-book 40-lakh-rupee print moved the market when it did not.
Limit Order Book Features You Can Build
From a snapshot feed with depth you can construct the standard set: the bid-ask spread, depth at best five levels, the depth imbalance (bid size divided by total depth), and the weighted average price across the top levels. Add the volume-weighted order flow, a running sum of signed trade volume, and the microprice, which weights the two touch prices by the sizes queued behind them. None of these need exotic software; each is a rolling calculation on a pandas frame. On a liquid underlying like Bank Nifty the depth imbalance beats raw volume for signalling the next few ticks of pressure.
Classifying Aggression: Trade Direction
Individual trade prints carry no buy or sell flag, so the model must infer direction by comparing each print to the prevailing quote: a trade at the ask is aggressive buying, at the bid aggressive selling, and inside the spread must be time-matched to the nearest quote state. This classification is the foundation of every flow feature, and it fails when timestamps are misaligned. Get the trade and quote stream on the same clock before the model sees data. A 20 percent misclassification in trade direction is enough to erase the signal value of imbalance features within a single day's training window.
Liquidity Imbalance as a Forecasting Input
A workable modelling recipe normalises the signed volume flow by the total volume over a rolling 5-minute window and feeds it to a gradient-boosted model alongside the microprice and spread. The feature family separates news-driven jumps from genuine aggression: a large unfair volume spike without depth change is ordinary liquidity absorption, while a moderate flow with a widening spread signals an informed trader paying for urgency. Forecast the midpoint's next 10-tick direction and evaluate with a market-order fill model, because a microstructure edge quoted inside the spread disappears once the spread is your entry cost.
Backtesting Microstructure Signals Without Scope Creep
Start with one instrument and one hour of the day, typically the first half-hour after open when Indian volume peaks, and record every backtest decision at the then-current quotes. Then measure decay: recompute the model's hit rate with a one-bar delay in the signal, and reject the strat if a 30-minute staleness cut takes it below a profitable line. Microstructure edges are the fastest-decaying edges that exist, so design the backtest to reward signal that is actionable now, not one-tick-perfect in a replay client.
- Deduplicate ticks and align clocks before any feature work.
- Classify trade direction against the prevailing quote.
- Normalise flow by volume and window the features.
- Backtest at the prevailing spread, not the theoretical mid.
- Test signal decay before you fund a live feed.
Fee-Aware Microstructure and the Data Bill
Microstructure edges quote inside the spread, and the edge's size after execution costs is the only size that matters: a signal that predicts the next ten ticks at 58 percent earns nothing if the round-trip spread and exchange charges eat more than the forecast adds. Price the data honestly too - tick and depth history from a feed vendor costs real money per instrument per month - and split the project budget into the data line, the compute line, and the exchange-cost line before the first model trains. The professional gate is the per-trade net after fees computed from the model's actual cleaned fills, not the theoretical mid; when that net stays negative across a two-week paper window, the signal is too slow or too thin for the venue that charges it.
