Modern Portfolio Theory

Harry Markowitz's framework for optimal portfolio construction balancing risk and return.

Key Concepts

  • Efficient frontier
  • Minimum variance portfolio
  • Maximum Sharpe ratio portfolio
  • Risk-free rate

Python Implementation

  • Calculate expected returns
  • Compute covariance matrix
  • Optimize using SciPy
  • Visualize efficient frontier

Example Code

from scipy.optimize import minimize
def portfolio_stats(weights, returns, cov):
    ret = np.sum(returns * weights)
    std = np.sqrt(np.dot(weights.T, np.dot(cov, weights)))
    return ret, std

From Mean-Variance Theory to Executable Code

Modern portfolio theory, introduced by Harry Markowitz, frames investing as a trade-off between expected return and risk, and finds the asset weights that deliver the best return for a given level of risk. The efficient frontier is the set of portfolios that dominate all others, and optimisation finds a point on that frontier. Python, with SciPy's optimisation routines, turns this elegant theory into code that works on real Indian asset returns.

The optimisation needs three inputs: expected returns for each asset, the covariance matrix describing how they move together, and a risk measure such as variance or standard deviation. From these, SciPy solves for the weight vector that either maximises return for a target volatility or minimises volatility for a target return, subject to constraints such as weights summing to one and no short selling.

Setting Up the Input Data

Collect daily or monthly returns for the assets, such as Nifty, Bank Nifty, sovereign bonds, gold and a small-cap index, over several years. Compute the mean return and the covariance matrix, using the sample covariance or a more stable estimator. Because assets with different scales distort the variance computation, work with percentage returns throughout. Clean the data for dividends and corporate actions so the returns reflect actual performance of a holding strategy.

Defining the Optimisation Problem in SciPy

SciPy's minimize function solves the optimisation when given an objective, an initial guess and constraints. For a minimum-variance portfolio, the objective is the portfolio variance computed as the weight vector times the covariance matrix, and the equity constraint forces the weights to sum to one, with bounds keeping each weight positive for a long-only portfolio. The result is the set of weights that produce the lowest possible volatility given the assets.

A Worked Example

  • Load 10 years of monthly returns for Nifty and gold.
  • Compute the mean vector and covariance matrix.
  • Define a function that returns portfolio variance from weights.
  • Call minimize with the variance objective and a sum-to-one constraint.
  • Record the optimal weights and the resulting expected return and volatility.

Building the Efficient Frontier

By sweeping across target returns, you can trace the entire efficient frontier, the curve of best performing portfolios, and select the point that matches your risk appetite. A risk-averse investor picks the lower-left portion of the curve; a risk-seeking one takes a higher-return, higher-volatility point. The tangent portfolio, where a line from the risk-free rate touches the frontier, maximises the Sharpe ratio and is a classic choice for an aggressive allocation.

Practical Pitfalls and Fixes

  1. Sample means are notoriously noisy estimates of future returns; shrink or equalise them.
  2. Covariance estimates overfit the sample; use a shrinkage estimator or factor model.
  3. Optimised weights are unstable, swinging wildly with small data changes; add a weight constraint or resample.
  4. Never forget transaction costs when rebalancing to the optimal weights.

Moving Beyond Markowitz with SciPy

SciPy's flexibility extends beyond plain variance minimisation. Risk parity weights, which equalise risk contribution, reduce to a non-linear system solved with root-finding. Maximum-drawdown and downside-risk optimisations substitute a more realistic risk measure for variance. Black-Litterman blending lets you combine market-implied returns with your own views. Each extension is a few lines of SciPy code around the same core solver, turning Markowitz's framework into a customised tool for the specific risk an investor wants to control.

The winning habit is to treat optimisation output as a starting point, not a final answer. Validate the weights on a holdout period, add realistic constraints and costs, and re-optimise periodically as the market changes. Used critically, SciPy turns a textbook theory into a disciplined, data-driven framework for building and maintaining a portfolio that matches your true risk tolerance.

Debugging Optimizer Runs

Confirm the covariance estimate is positive semi-definite before passing it to the optimizer, then solve on annualised data under a Sharpe maximisation objective. Compare the Kelly-optimal fraction against the optimizer's variance floor to avoid over-betting a noisy edge. Read any 'infeasible' status as a data problem first, and simplify constraints before blaming the algorithm.

Feeding the Optimizer Real Indian Data

SciPy will happily return weights from whatever you give it, so the quality of the input decides the quality of the answer. Pull historical daily returns for the assets you actually can trade here, such as Nifty, Bank Nifty, a gilt index, gold and a mid-cap fund, and align them on matching dates so the covariance matrix is not distorted by mismatched calendars. Prefer a shrinking or factor-based covariance estimator over the raw sample, since the raw version overfits a short window, then sweep the target-return grid to trace an efficient frontier you recognise as realistic. Remember that expected-return estimates are guesses, so stress the weights by re-running the optimisation under a few different return assumptions and by holding back the most recent year for validation. Finally, add a transaction-cost penalty before you rebalance, because paying brokerage and STT to chase a tiny frontier improvement is how a theoretically optimal plan quietly underperforms a simple one.