From befcc3370279d4ee716c58ff347ad2a819c0c19d Mon Sep 17 00:00:00 2001 From: zhaoli Date: Thu, 20 Aug 2026 05:09:13 +0000 Subject: [PATCH] Q14 result: compact stochastic set REFUTED on single-stock universe (EVIDENCE#033, exp 50) --- book/CLAIMS.md | 3 ++- book/EVIDENCE.md | 1 + 2 files changed, 3 insertions(+), 1 deletion(-) diff --git a/book/CLAIMS.md b/book/CLAIMS.md index 24582f4..24da2f4 100644 --- a/book/CLAIMS.md +++ b/book/CLAIMS.md @@ -23,6 +23,7 @@ The running scoreboard of every quantitative claim in the book. Updated per chap | Baseline 1-day LGB signal is weak / costs erase most of the edge | HYPOTHESIS (idea: pre-clean-lake, exp 8) | EVIDENCE#001/002 → exp 8 | | More features ≠ better signal on a small (50-name) cross-section | HYPOTHESIS (3+ supporting runs, panel-specific) | EVIDENCE#003/004/014/017/019 | | Mean reversion (OU z-score, trend-slope reversal) is the stable single-feature edge | REFUTED (single-feature trend-slope reversal tested, no reversal learned) | EVIDENCE#032 → exp 43 (Q11) | +| Compact stochastic set generalizes to liquid single-stock names | REFUTED (out-of-universe RankIC −0.02, ICIR −0.07 — signal is noise on 30-name stock panel) | EVIDENCE#033 → exp 50 (Q14) | | Assets are submartingales long-horizon / mean-reverting short-horizon (VR<1 at 5–20d) | HYPOTHESIS (chat-derived martingale study, exp 19 never closed) | book/data/chat_mining/martingale-study.txt | ## Model @@ -80,5 +81,5 @@ The running scoreboard of every quantitative claim in the book. Updated per chap - exp 15 Kelly sizing: re-run — DONE — refuted on the clean lake by Q06 (exp 38); mark the old hypothesis REFUTED. - exp 18 risk-limit spec: re-validate $5M liquidity floor on the post-reset reference signal — DONE — refuted as an IR lever by Q08 (exp 40); keep as safety net only. - Weekly rebalance: reproduce on a second window / take to a live round. -- Out-of-universe validation: non-ETF universe for the compact stochastic feature set. +- Out-of-universe validation: non-ETF universe for the compact stochastic feature set. — DONE — refuted by Q14 (exp 50); RankIC −0.02, ICIR −0.07 on 30 liquid single-stock names. - Long-horizon label (10d/22d) with a matching low-turnover construction (e.g. weekly recompute) — signal says the edge is there, cost says daily churn kills it; untested combination. \ No newline at end of file diff --git a/book/EVIDENCE.md b/book/EVIDENCE.md index ae3ffd9..6813ed1 100644 --- a/book/EVIDENCE.md +++ b/book/EVIDENCE.md @@ -49,6 +49,7 @@ Experiments 8–18 record metrics under a legacy schema (`ls_sharpe`, `maxdd_wit | EVIDENCE#030 | Q09 long-short top10/bottom10: real pre-cost edge (gross +6.57%, IR 0.656) destroyed by daily L/S turnover — total_cost $96,721 (≈9.7% of $1M), 2485 trades/150d, fill rate 0.401; net −8.38%, IR −0.834, maxDD −11.22%. | exp 41, run `0647eadd…` (mlflow exp 39), branch `exp/41-q09-long-short-market-neutral-long-top-1` | yes — Q09 FAIL (turnover kills) | | EVIDENCE#031 | Q10 HMM regime entry gate (sp_hmm_p_regime1 ≥ 0.5 overlay): meets only the DD leg (−7.38% maxDD) — churns 276 trades/150d, cost ~6.3pp erases +2.02% gross; net −4.26%, IR −0.382. Regime-overlay hypothesis refuted. | exp 42, run `436acd01…` (mlflow exp 40), branch `exp/42-q10-hmm-regime-overlay-entry-gate-on-sph` | yes — Q10 FAIL | | EVIDENCE#032 | Q11 standalone 5d reversal (single feature sp_trend_slope_5): IC is slightly positive (+0.0023), so the model did NOT learn reversal — the pooled trend-slope reversal beta does not reproduce standalone. Gross −10.36%, net −15.22% (IR −1.572). Cost is not the culprit. | exp 43, run `e859adfe…` (mlflow exp 41), branch `exp/43-q11-standalone-5d-reversal-single-featur` | yes — Q11 FAIL (no reversal learned) | +| EVIDENCE#033 | Q14 out-of-universe validation: compact stochastic set on 30 liquid single-stock names (AAPL,MSFT,NVDA,…). RankIC −0.0198 (needed >0.03), ICIR −0.073 (needed >0.15) — signal is noise on this universe. Net P&L positive (+10.02% ann, IR 0.668, maxDD −6.67%) but that is top-10 concentration luck, not predictive signal. Train RankIC 0.316 shows the model overfits to the 50-ETF panel. | exp 50, run `809ff460…` (mlflow exp 50 `tac-rd-q14-out-of-universe`), branch `exp/50-q14-compact-stochastic-set-generalizes-t` | yes — Q14 FAIL (signal does not generalize cross-universe) | ## Live execution trail