Simulation tests

Stress tests for a return stream: Monte Carlo resampling (fan chart and outcome distributions), variance testing (does the edge survive win-rate drift?), and noise testing (does it survive magnitude jitter?). These code cells are executed live by Quarto every time the docs are built.

Sample data

We reuse qrt’s bundled sample datasets — AAPL as the “strategy” and SPY as the benchmark — loaded offline via q.data.datasets.load, no network dependency:

import pandas as pd

import qrt as q

aapl = q.data.datasets.load("aapl")
spy = q.data.datasets.load("spy")

returns = pd.concat(
    {
        "AAPL": aapl["close"].pct_change(),
        "SPY": spy["close"].pct_change(),
    },
    axis=1,
).dropna()
strategy = returns["AAPL"]
benchmark = returns["SPY"]

returns.tail()
AAPL SPY
datetime
2026-07-13 0.006311 -0.007656
2026-07-14 -0.007721 0.003551
2026-07-15 0.040145 0.003964
2026-07-16 0.017588 -0.005419
2026-07-17 0.001440 -0.009897

Monte Carlo fan chart: q.plot.montecarlo

Runs q.stats.montecarlo — which resamples historical returns with replacement (a bootstrap), not a plain permutation, since permuting a fixed set of returns can’t change their compounded terminal value — and renders a sample of the simulated paths as a fan chart around a shaded confidence band. Paths are colored by outcome relative to the optional bust/goal thresholds, and the original (unresampled) path is highlighted on top.

By default each period is resampled independently (i.i.d.), which assumes no serial dependence between returns — an assumption real markets violate through volatility clustering and autocorrelation. Passing block_size switches to a stationary block bootstrap that resamples contiguous, randomly-lengthed blocks of history instead of individual days, preserving that local structure.

We also decouple the simulation horizon from the resampling pool: returns is the full AAPL history (avoiding bias toward a single market regime), but periods=252 simulates only one trading year forward. Without this, compounding variance over the full ~26-year history balloons the terminal-value spread to unreadable extremes and saturates bust/goal probabilities near 0%/100% (a long enough random path is nearly guaranteed to cross any fixed threshold eventually) — real effects, but a poor, uninformative simulation:

fig = q.plot.montecarlo(
    strategy,
    sims=1000,
    periods=252,
    bust=-0.3,
    goal=0.5,
    seed=7,
    sample=150,
    block_size=20,
    title="AAPL Monte Carlo simulation (1-year horizon, block bootstrap)",
)
fig.show()

Monte Carlo distribution: q.plot.montecarlo_distribution

Complements the fan chart with two histograms built from the same (block bootstrap, 1-year horizon) simulations: the spread of terminal returns (reward) alongside the spread of each simulation’s Max Drawdown (risk) — surfacing the downside risk a terminal-value histogram alone would hide. The original outcome and the bust/goal thresholds are marked on each panel:

fig = q.plot.montecarlo_distribution(
    strategy,
    sims=1000,
    periods=252,
    bust=-0.3,
    goal=0.5,
    seed=7,
    block_size=20,
    title="AAPL Monte Carlo outcome distribution (1-year horizon, block bootstrap)",
)
fig.show()

Variance Testing fan chart: q.plot.variance_test

Runs q.stats.variance_test, which simulates forward, trade-by-trade paths rather than resampling historical periods: each simulated path draws periods future trades using a per-simulation win rate randomly perturbed by win_rate_variance from the historical win rate, then samples win/loss trade returns with replacement accordingly. This stresses whether a strategy’s edge holds up if its win rate drifts, rather than just reshuffling the exact historical sequence — complementing q.plot.montecarlo’s “same edge, different order/draws” view with a “what if the edge itself drifts” view.

Paths are colored by outcome — ruined (Max Drawdown breached ruin) in red, profitable (positive terminal return) in green, neutral in gray — with the actual historical cumulative return ("Real") drawn in bold black for comparison. Like q.stats.montecarlo, results are expressed in compounded returns, not account-currency dollars. Since strategy has a DatetimeIndex, the x-axis shows real calendar dates: the actual dates for "Real", then dates extrapolated forward from the last one for the simulated paths (marked with a dotted “Now” line at the boundary) — matching the date axes used elsewhere on this page. Here we treat each AAPL trading day as a stand-in for a discrete trade:

fig = q.plot.variance_test(
    strategy,
    periods=252,
    sims=1000,
    win_rate_variance=0.1,
    ruin=-0.5,
    seed=7,
    sample=150,
    title="AAPL Variance Testing (1-year horizon)",
)
fig.show()

Noise Test fan chart: q.plot.noise_test

Runs q.stats.noise_test, which keeps the exact historical sequence of returns (same order, same length, same dates) but jitters each period’s return magnitude with multiplicative noise (1 + eps, eps ~ Normal(0, noise)), never flipping its sign. This tests whether a strategy’s performance is sensitive to day-to-day noise/volatility in the underlying data — a different question from q.plot.montecarlo’s “same returns, different resampled order/draws” or q.plot.variance_test’s “same win rate distribution, forward horizon” views.

Paths are colored neutral gray ("Random") by default, or by the optional bust/goal thresholds when given, with the original, unperturbed cumulative return ("Real") drawn in bold black for comparison. Since no resampling or forward extrapolation happens, the x-axis is simply returns’s own dates and the y-axis is compounded percentage return, not price:

fig = q.plot.noise_test(
    strategy,
    sims=1000,
    noise=0.1,
    seed=7,
    sample=150,
    title="AAPL Noise Test",
)
fig.show()
Back to top