book: add ch 11 walk-forward + 5 refuted guards (exp 52-56); EVIDENCE#043-047; rename 12→13-synthesis

2026-08-20 21:03:59 +00:00
parent c0d21115eb
commit 18b61bbb8f
8 changed files with 144 additions and 19 deletions
@@ -8,7 +8,7 @@ The metrics vocabulary of ch. 01 is only useful if the numbers can be trusted. T

 1. **Lake** — bars and features live in one hive-partitioned lake (market/timeframe/symbol, `family=ta|sp`), with coverage and calendar metadata. It is the single source of bar/feature truth. The clean-lake rebuild proved the stakes: when the lake was rebuilt, the reference signal collapsed (IC 0.0354 → 0.0019) because the old lake's data quality had silently inflated results `(PROVEN → exp 21)`. A claim built on the lake is only as good as the lake.
 2. **Experiment** — every run is a traced experiment: a git branch (`exp/N-…`), an MLflow run with recorded config/params/metrics, and hypothesis/evaluation notes recorded before and after the run. The traceability loop was used on exp 8–31; the branch, run, and notes are the reproducible unit `(PROVEN → the traced experiment store; see `rd_exp_*` tools and `EVIDENCE.md`)`.
-3. **Live** — a proved book advances to a round window (targets → intents → decisions → orders → fills), and is reconciled (slippage bps, cost, funnel) `(PROVEN → round 3; tac-rd-book trail)`. Live beats backtest: a claim about trading performance must trace to a round, not to a backtest (ch. 11).
+3. **Live** — a proved book advances to a round window (targets → intents → decisions → orders → fills), and is reconciled (slippage bps, cost, funnel) `(PROVEN → round 3; tac-rd-book trail)`. Live beats backtest: a claim about trading performance must trace to a round, not to a backtest (ch. 12).

 ## Why traceability is the methodology

@@ -29,6 +29,6 @@ Most additions failed. The loop's value is not that it produced winners — it i

 ## From here

-Ch. 03 applies the loop to the book's first worked question (does the signal clear costs?), ch. 06 to the clean-lake reset, and ch. 11 to the live round that closes the loop with reconciliation.
+Ch. 03 applies the loop to the book's first worked question (does the signal clear costs?), ch. 06 to the clean-lake reset, ch. 11 to walk-forward re-validation of the campaign's headline results, and ch. 12 to the live round that closes the loop with reconciliation.

 Open questions: purge/walk-forward CV instead of single train/valid split `TODO(evidence-needed: purged CV on the exp-26 reference)`, and a PSI-based drift-aware retraining gate `(HYPOTHESIS → chat-ideas.md)`.