Survivorship Bias in Public Trader Rankings — A Quantitative Analysis
1. Setup
The structural problem is straightforward. A ranking site lists the top-N traders by some performance metric (return, Sharpe, ELO). To enter the list, a trader must have a current track record. If a trader's track record ends — for any reason, including but not limited to a catastrophic loss — they fall off the list. The remaining list, by construction, over-represents survival-conditional performance.
This is a well-understood phenomenon in mutual-fund analysis (Brown et al., 1992; Carhart, 1997) and hedge-fund performance studies (Malkiel and Saha, 2005). The contribution of this paper is to estimate its magnitude in the specific case of public trader-ranking sites — a category whose evidence base is more uneven than mutual funds because audit grade varies dramatically across operators.
2. Synthetic universe construction
The worked example constructs a 14-year synthetic universe (2012-2025 inclusive) of 5,000 simulated traders. Return assumptions are plausible scenario inputs informed by broad ranges discussed in public competition results and survivorship-bias literature; they are not calibrated to a private audit dataset and do not represent audit-grade ground truth.
The universe includes a mortality term: in each year, each trader has a probability of exit conditional on their drawdown. Traders with maximum drawdown above 50% in a given year exit at 38% probability; above 30%, at 14%; below 30%, at 1.7%. These are illustrative attrition assumptions chosen to show sensitivity to drawdown-linked exits; they are not estimates from a QTJ audit-record database.
3. Estimating the bias
The bias estimator is the difference between (a) the cohort-inclusive mean of all 5,000 simulated traders' returns, and (b) the survival-conditional mean restricted to traders with active records at the end of the window.
B(T) = E[R | active at T] − E[R | cohort-inclusive]
We compute B(T) for windows of 1, 3, 5, 7, 10, and 14 years. The Monte Carlo simulation runs 10,000 iterations and reports the mean and 90% confidence interval.
| Lookback (years) | Survival-conditional mean | Cohort-inclusive mean | Bias (bps) | 90% CI |
|---|---|---|---|---|
| 1 | +12.6% | +12.4% | +22 | (+15, +29) |
| 3 | +13.2% | +12.4% | +82 | (+71, +94) |
| 5 | +13.9% | +12.4% | +148 | (+131, +166) |
| 7 | +14.6% | +12.4% | +220 | (+196, +245) |
| 10 | +15.2% | +12.4% | +277 | (+248, +307) |
| 14 | +15.8% | +12.4% | +339 | (+304, +376) |
The bias scales approximately linearly with the lookback window, around +30 bps per additional year of historical inclusion in the cohort. This is consistent with mutual-fund survivorship literature (Carhart 1997 found ~70 bps per year for equity funds; the lower number here reflects the higher dispersion of public trader returns relative to mutual funds, which spreads the survivorship effect across more contributing variance).
4. Audit-grade matters
The simulation also tests the effect of audit grade. We re-run the analysis under three scenarios:
Scenario A: Audit-grade tracking — failed traders' final-year returns remain in the database. Bias at 7 years: +89 bps.
Scenario B: Mixed audit-grade and self-reported — failed traders' records are dropped at exit. Bias at 7 years: +220 bps (the headline number above).
Scenario C: Pure self-reported — failed traders' records are dropped, AND surviving traders may revise prior-year returns upward (small positive lognormal noise applied to the prior-year return distribution). Bias at 7 years: +413 bps.
Scenario A represents a higher-audit-coverage assumption, while Scenario C represents a predominantly self-reported universe with mild upward revision. The implication for consumers of public trader rankings is straightforward: prefer audit-grade tracking, since the bias is substantially smaller, and discount displayed mean returns by 200-400 basis points when reading rankings whose underlying tracking is mixed or self-reported.
5. Recommended correction
The cohort-inclusive Bayesian shrinkage correction is well-known: shrink the survival-conditional mean toward the cohort-inclusive prior with a weight set by the observed mortality rate. In code-form:
corrected_mean = (1 - w) * survival_mean + w * (cohort_prior - mortality_correction)
The weight w can be set empirically from the historical mortality rate within the database, defaulting to ~0.4 for 7-year windows under audit-grade tracking and ~0.55 for 7-year windows under mixed or self-reported tracking. A worked example with code is available on request from the editorial team.
6. Implications for the trader-ranking literature
Three implications. First, ranking sites that include audit-grade tracking with explicit cohort recording (Tier 1 broker-statement audits, the WCTC official results, BarclayHedge fund-tracked records) produce substantially less biased estimates than self-reported rankings. Second, the historical lookback window matters more than the methodology — a 14-year self-reported ranking will exhibit larger bias than a 1-year audit-grade ranking, regardless of how sophisticated the survival-conditional methodology is. Third, the corrected estimates produce different conclusions about who the top performers actually are: the difference between Tier 1 audited mean returns and self-reported mean returns shrinks substantially after the bias correction.
The framing draws on public competition results, published survivorship-bias literature, and public trader-ranking methodology pages. The simulation inputs remain illustrative.
References: full reference list on request via the QTJ editor inbox.