Data Sources
- NSE website CSV downloads
- nsepy library
- YFinance for Indian stocks
Analysis Techniques
- Historical price analysis
- Options chain analysis
- Volume analysis
Python Code
import nsepy
from nsepy import get_history
symbol = 'RELIANCE'
data = get_history(symbol, start=date(2020,1,1))Applications
Custom screening, backtesting, and signal generation.
Setting Up Your NSE Data Environment
Analysing NSE market data with Python requires three moving parts: a data source, a clean dataframe, and a broker or exchange API for live updates. For historical daily bars, the official NSE bhavcopy CSV published after each session is the most reliable free source; for intraday data, you often need a paid vendor or the broker's own SDK because NSE does not publish tick files publicly.
Install pandas, numpy, requests and matplotlib in a virtual environment. Keep an eye on the NSE website's rate limits; hammering the CSV endpoint hundreds of times per minute will get your IP transiently blocked. A simple sleep of one second between downloads stays inside the tolerated band.
Reading the Bhavcopy into a DataFrame
The standard bhavcopy CSV has columns for symbol, series, open, high, low, close, last, prevclose, total traded quantity, and turnover. Loading it with pd.read_csv() and normalising symbols to uppercase gives you a baseline frame. Add corporate-action-adjusted prices from the capital-market file before doing any percentage move analysis, otherwise dividend dates will create phantom drops in your series.
Computing Technical Signals in Bulk
Vectorised pandas operations let you compute indicators for the entire index of stock history at once. A rolling 20-day mean with smoothed.rolling(20).mean() and a rolling standard deviation give you a crude Z-score signal; the same window powers a simple breakout detector when the close exceeds the 20-day high. All of it runs in milliseconds, which is what makes Python the right tool for scanning 1,800 symbols nightly.
- Fetch the daily close for the full universe.
- Compute rolling mean, std and 20-day high/low in one pass.
- Merge the option-chain implied-volatility column if you go intraday.
- Rank symbols by momentum score and dump the top 25 to a CSV.
Option Chain Data Handling
The NSE option chain JSON returns strikes, open interest and implied volatility for both calls and puts. Flatten the nested JSON with json_normalize, align expiry dates, and pivot so strikes become columns with IV on the cells. This pivot is the raw material for smile curves; plotting the IV against strike for a single expiry is the standard diagnostic before any option strategy.
Common Pitfalls and Fixes
- Timezone drift: NSE times are IST; always convert timestamps explicitly or date filters silently return empty frames.
- NaN padding: New listings and trading halts leave NaN rows; fill with forward-fill only where economically meaningful.
- Corporate actions: Apply the adjustment factor before any percentage-change math.
- Look-ahead bias: Never use data from the same bar as the signal to make today's decision.
When you finally connect a live SDK, keep the downloaded history as your backtest set and the streaming feed purely for order management; mixing the two is the single most common engineering mistake in retail quantitative setups.
Fetching the Nifty Options Chain Publicly
NSE exposes a free JSON endpoint for the live option chain that lists every strike with spot price, open, high, low, closing price of the index, and the full call and put columns. A simple requests call with the right headers returns this in seconds, and the same pattern pulls historical EOD bhavcopy files. There is no official guarantee and the endpoint is throttled, so cache every response to disk: fetch once, reuse for the rest of the research day, and schedule a cron-based refresh rather than hammering the endpoint in a loop. Confirmed working patterns use a browser-like User-Agent and respect the exchange's session rate trust.
Session Rate Handling and Banned Fields
Rate limits on the free endpoint punish the impatient: a tight loop for many strikes or many symbols triggers temporary hardening. Buffer every fetch with a small sleep and batch requests to run sequentially at intervals. Some bhavcopy fields come back as strings or with non-standard characters; normalise the symbol column, strip currency markers, and coerce the numeric columns with an explicit dtype map before any computation. Debugging an options pipeline that silently multiplied string premiums is a classically expensive hour.
Rebuilding OHLC From a Tick Feed
When you hold a raw tick stream, resampling to OHLC is a ~15-line pandas exercise: set the index to the exchange timestamp, resample by 1-minute or 15-minute frequency, and aggregate open, high, low, close, and volume in one call. The subtlety is volumes: sum them during the resample rather than taking the last value, and handle the first tick of each new bar explicitly so the open is the first trade's price, not the previous bar's close. This simple rebuild produces the exact feature table a daily model and an intraday model share upstream.
A Mini Feature Library in Thirty Lines
Justifiable space: build thirty lines that turn a closes frame into the entire starter library. Rolling returns of 1, 3, 5, and 20 days; rolling realised volatility as the standard deviation of daily returns over 20 days; a Relative Strength Index at 14; a rolling z-score of close against its 50-day mean; and the day-of-week counter. Each function computes inside a shift to avoid look-ahead, and the result concatenates into one feature frame. This library is the difference between a tutorial that ends at the charts and a permanent asset you reuse for every model in the future.
Deploying the Pipeline to a Schedule
Move the notebook logic into a single script that fetches, cleans, computes, and writes a dated CSV, then schedule it with cron or your broker server's task scheduler to run after market close. Version the output file by date so any later analysis can prove which data point the model had on any given Tuesday. A scheduled, versioned pipeline is the literal floor of reproducible quant work; every serious result is traceable to the exact dataset that produced it, and a python-cron loop from the NSE EOD feed is the cheapest way to build that floor.
- Cache responses; call the free endpoint on a schedule, not a loop.
- Coerce and normalise numeric fields before computing.
- Resample ticks with explicit open aggregation and summed volume.
- Expose a 30-line feature library with shift-protected inputs.
- Write dated CSVs and let cron run the refresh.
Corporate Actions and Calendar Alignment
A bhavcopy pipeline silently lies through corporate actions: a bonus, split, or dividend ex-date changes the price series, and an unadjusted file fuses the pre- and post-event prices into one misleading history. Apply the adjustment factor for the symbol before any feature spans the ex-date, and align everything to the exchange's own calendar so a holiday gap never becomes a genuine two-week feature. The alignment also governs the option chain: strikes and spot must map to the same session's snapshot, and the expiry countdown must count actual trading days left, not calendar days. A pipeline that adjusts actions and aligns calendars produces the one artifact every model needs - a history whose dates and prices tell the same honest story.
