zhaoli/tac-exp-dev
0
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

book: add perturbation stress test (EVIDENCE#049) — Config A 2026 robust to topk/n_drop/cost within window

Browse Source
This commit is contained in:
zhaoli
2026-08-20 21:44:04 +00:00
parent 18b61bbb8f
commit 4831a2770c
3 changed files with 58 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files
book/EVIDENCE.md
+1
View File
@@ -77,6 +77,7 @@ Experiments 8–18 record metrics under a legacy schema (`ls_sharpe`, `maxdd_wit
| ID | Claim | Source | Verified? |
|----|-------|--------|-----------|
| EVIDENCE#048 | Streaming IC circuit-breaker (`ic_min_rankic`, `ICGateTopkDropoutStrategy` in `tac_qlib/contrib/strategy/ic_gate.py`) trip-rate study: with thresholds 0.02–0.06, the gate trips on 25–50% of days in every year (2021–2026), freezing TopkDropout's rotation out of losers. A gate that trips every year cannot separate good years from bad. Do not deploy live. | ad-hoc scripted study on exp 52/53 pred/label artifacts, `tac_qlib/tac_qlib/contrib/strategy/ic_gate.py`, `tac_qlib/tac_qlib/risk_limits.py` | yes — guard candidate 3 REFUTED |
| EVIDENCE#049 | Perturbation stress test on Config A 2026 (exp 52, pred from run `9f98ea5c`): same signal, varying topk (5/10/15), n_drop (1/2/3), costs (base/high/5×base). **topk**: 10 optimal (32.8% raw, Sharpe 1.98); 5 loses ~0.5pp, 15 loses ~6.5pp. **n_drop**: 1 optimal; 2 loses ~6pp, 3 loses ~4pp. **costs**: immaterial — 5× cost increase (25bp/35bp/$15) drops return only 0.17pp (32.84%→32.67%). maxDD stable −5.8% to −7.0% across all perturbations. **Within the 2026 window the edge is robust to parameter perturbation.** The problem remains that it does not exist in other windows (ch 11). | ad-hoc rd_backtest grid on exp 52 pred.pkl, `book/data/perturbation/config_a_2026_sensitivity.json` | yes — within-window robustness confirmed |
## External references (book/references/)