Sharpe and Sortino answer how much reward was observed per unit of risk. PSR and PSoR answer a different question: how strong is the statistical evidence that the corresponding population ratio exceeds a chosen threshold?
That distinction matters because a large ratio from a short, skewed, or fat-tailed sample can be much less persuasive than the same ratio from a long, well-behaved sample. This notebook derives all four measures using QRT’s exact conventions, validates manual calculations against q.stats, and visualizes how confidence changes with the threshold.
The examples are educational diagnostics, not forecasts. None of these statistics removes selection bias, regime change, transaction costs, or dependence between observations.
Why visualize a probability curve?
A single PSR or PSoR value answers one question at one chosen threshold. For example, PSR @ Th = 0 asks whether the population Sharpe ratio is positive. The threshold curve answers the broader question: how does the evidence change as the required quality level increases?
The visualization is useful because it lets you:
assess robustness instead of relying on one headline probability;
compare several annualized ratio hurdles without recalculating each one manually;
find the largest threshold supported at a chosen confidence level, such as 95%;
distinguish a high observed ratio from strong statistical evidence;
compare confidence based on total volatility (PSR) with confidence based on downside risk (PSoR); and
see when a conclusion may depend on limited data, especially a small number of negative observations.
The X-axis is the annualized Sharpe or Sortino threshold. The Y-axis is the estimated probability that the corresponding population ratio exceeds that threshold. The curve must decrease from left to right because exceeding a more demanding hurdle is less likely.
1. Imports and reproducible return series
We use 252 observations per year and a fixed random seed. Four synthetic series isolate common distribution shapes; bundled AAPL returns provide a realistic, offline example. Keeping periods_per_year explicit prevents daily and annualized ratios from being mixed accidentally.
where \(s_x\) is the sample standard deviation (ddof=1). Sharpe is useful when upside and downside variability both represent risk, or when strategies have approximately symmetric return distributions. It is easy to compare, but can be distorted by outliers, negative skew, fat tails, stale prices, serial correlation, a mismatched risk-free rate, or an inappropriate annualization frequency. Set smart=True to apply QRT’s autocorrelation penalty to the denominator.
This is useful when positive volatility should not be penalized. QRT uses the deannualized rf as \(T_p\), squares the clipped downside for all\(n\) observations (zeros included), and uses no Bessel correction. This differs from implementations that take the standard deviation of only negative returns. The rf/minimum-acceptable-return changes which observations count as downside; it is separate from the PSoR ratio threshold introduced later. Sortino can still be unstable when downside observations are scarce or dominated by a few losses.
Sharpe and Sortino are point estimates: they describe this sample as if the estimated ratio were known exactly. Their uncertainty depends on sample size and return shape. More observations usually tighten inference; negative skew and excess kurtosis indicate asymmetric or unusually frequent extreme outcomes that make normal-theory confidence less persuasive.
PSR and PSoR turn a point estimate into a probability relative to an explicit benchmark. They do not mean “probability the strategy will be profitable next year.” They estimate confidence that the population risk-adjusted ratio exceeds the selected ratio threshold, conditional on the model and sample.
distribution_diagnostics = pd.DataFrame( { name: {"Observations": len(series),"Mean": series.mean(),"Volatility": series.std(ddof=1),"Skewness": series.skew(),"Excess kurtosis": series.kurtosis(), }for name, series in {**synthetic_returns, "AAPL": returns}.items() }).Tdistribution_diagnostics
Observations
Mean
Volatility
Skewness
Excess kurtosis
Normal
756.0000
0.0000
0.0119
0.0321
-0.0791
Negative skew
756.0000
-0.0067
0.0088
-3.5495
21.1854
Fat tailed
756.0000
-0.0002
0.0116
-0.1735
2.9131
Frequent gains / rare losses
756.0000
0.0006
0.0121
-6.0796
35.0546
AAPL
2653.0000
0.0011
0.0182
0.1331
6.7656
5. Probabilistic Sharpe Ratio (PSR)
For an annualized benchmark \(SR^*_{ann}\), QRT first converts both the observed ratio and threshold to periodic units:
Let \(n\) be the full sample size, \(\gamma_3\) sample skewness, and \(\gamma_4\) raw kurtosis (Pandas excess kurtosis plus 3). QRT implements the Bailey and Lopez de Prado adjustment
A PSR near 1 is stronger in-sample evidence that Sharpe exceeds the threshold; it is not a guarantee of future performance. At a threshold equal to the observed Sharpe, the numerator is zero, so \(PSR=\Phi(0)=50\%\).
Annualized Sharpe 1.0018
Annualized threshold 0.0000
PSR probability 0.9994
PSR percentage 99.94%
dtype: object
6. Probabilistic Sortino Ratio (PSoR)
PSoR estimates \(P(SoR>SoR^*)\) and is the confidence-aware counterpart to Sortino. Two thresholds must not be confused:
rf is the annualized minimum acceptable return used to construct excess returns and downside observations.
threshold is the annualized Sortino-ratio benchmark that the unknown population Sortino should exceed.
QRT converts observed Sortino and \(SoR^*\) to periodic units. It then keeps only negative excess returns, giving downside count \(n_d\) and downside skewness \(\gamma_3^-\). Its downside-specific approximation is
Unlike PSR, QRT’s PSoR does not use full-sample kurtosis. It returns NaN when fewer than three downside observations are available or the estimated variance is not positive. Because \(n_d\) may be much smaller than \(n\), PSoR can express substantially more uncertainty than a large Sortino point estimate suggests.
The public APIs accept a Pandas return Series and return a Python float probability in \([0,1]\) when defined. Both threshold arguments are annualized ratio thresholds and default to 0; rf is an annualized rate and also defaults to 0. periods_per_year=None infers frequency from the index, while this notebook passes 252 explicitly.
The assertions below protect against a hidden unit mismatch: manual and library results must agree, and a threshold equal to the observed annualized ratio must produce 50%.
The same mean and volatility do not imply the same evidence. PSR responds to full-sample size, skewness, and kurtosis; PSoR responds to the number and skewness of downside observations. Sortino can greatly exceed Sharpe for a stream with many small positive moves, but PSoR should temper that point estimate when only a small or highly skewed downside sample supports it.
A threshold is the hurdle ratio, not a return target. threshold=0 asks whether the population ratio is positive; threshold=1 asks whether it exceeds an annualized ratio of 1. Raising the hurdle must lower the probability.
When reading either curve, inspect five features:
Height at a chosen threshold. A value of 90% at threshold 1 means the estimated probability is 90% that the population ratio exceeds an annualized ratio of 1.
Slope. A slowly declining curve indicates that the conclusion remains relatively robust as the hurdle rises. A steep curve means the result is sensitive to the selected threshold.
Intersection with the 95% line. This approximates the largest ratio threshold supported with 95% confidence. It is often more informative than reporting only the observed ratio.
Observed-ratio line. At a threshold equal to the observed ratio, probability is 50% because the sampling distribution is centered on the estimate. This is expected, not a warning sign.
Value at threshold 0. This is the evidence that the population risk-adjusted ratio is positive. A high value at zero does not imply similarly strong evidence that the ratio exceeds 1 or 2.
Use q.plot.psr and q.plot.psor for standalone charts. Both X-axes are annualized, matching the public statistics APIs and the report cards.
thresholds = pd.Index([0.0, 0.5, 1.0, 1.5, 2.0], name="Annualized ratio threshold")threshold_probabilities = pd.DataFrame( {"PSR": [ q.stats.probabilistic_ratio( returns, threshold=value, periods_per_year=periods_per_year )for value in thresholds ],"PSoR": [ q.stats.probabilistic_sortino_ratio( returns, threshold=value, periods_per_year=periods_per_year )for value in thresholds ], }, index=thresholds,)assert threshold_probabilities.diff().iloc[1:].le(0.0).all().all()threshold_probabilities.style.format("{:.2%}")
PSR
PSoR
Annualized ratio threshold
0.000000
99.94%
99.83%
0.500000
94.82%
97.37%
1.000000
50.24%
82.90%
1.500000
5.31%
48.50%
2.000000
0.06%
15.26%
psr_figure = q.plot.psr( returns, periods_per_year=periods_per_year, threshold_max=3.0, title="AAPL: probability that Sharpe exceeds threshold",)psr_figure.show()
Reading the PSR curve
Start at a Sharpe threshold that represents the strategy’s intended use. Threshold 0 tests whether risk-adjusted performance is positive; threshold 1 is a more demanding quality hurdle.
For this graph, observe:
how far right the blue curve remains above the 95% guide;
whether probability collapses quickly as the threshold rises; and
the distance between threshold 0 and the observed Sharpe line.
A high PSR @ Th = 0 with a rapid decline before threshold 1 means there is strong evidence of a positive Sharpe, but much weaker evidence of an excellent Sharpe. A flatter, right-shifted curve supports stronger thresholds. Two strategies can have the same observed Sharpe but different PSR curves because sample size, skewness, and kurtosis affect the uncertainty of the estimate.
psor_figure = q.plot.psor( returns, periods_per_year=periods_per_year, threshold_max=3.0, title="AAPL: probability that Sortino exceeds threshold",)psor_figure.show()
Reading the PSoR curve and comparing both views
Read the orange PSoR curve in the same way, but remember that its uncertainty is driven by the negative excess-return observations. A small downside sample or strongly skewed losses can make PSoR less decisive even when the observed Sortino ratio is high.
Comparing the two curves can reveal the strategy’s risk shape:
PSoR below PSR: downside performance is less firmly supported, often because there are few losses or those losses are strongly skewed.
PSoR above PSR: much of the total variability may be positive variation, which Sharpe penalizes but Sortino does not.
Similar curves: total-volatility and downside-risk views provide broadly consistent evidence.
When comparing strategies, do not rank them only by the probability at threshold 0. Compare probabilities at economically meaningful thresholds and inspect curve shape. A curve that stays higher farther to the right provides stronger evidence across demanding hurdles.
These are in-sample confidence statements, not forecasts. They do not correct for testing many strategies, choosing the best result after inspection, regime changes, costs, or dependent observations. Out-of-sample validation and selection-bias controls remain necessary.
10. Cards must name their threshold
A probability without its benchmark is ambiguous. QRT’s backtest report computes both headline probabilities at the annualized ratio threshold 0, so the cards are labeled exactly PSR @ Th = 0 and PSoR @ Th = 0. That means “probability the population ratio is positive.”
When building custom cards, keep each threshold and label in the same configuration object. Changing the threshold then updates both the calculation and visible title.
QRT drops missing values, rejects empty/non-numeric/infinite inputs, and rejects simple returns below -100%. Sharpe and Sortino are undefined (NaN) when their denominator is zero. PSR also needs at least two valid observations and a positive finite variance estimate. PSoR is deliberately stricter: it needs at least three negative excess-return observations.
The table below records outcomes without hiding exceptions. A finite probability must remain in \([0,1]\); NaN means the estimate is undefined rather than zero confidence.