tac-exp-dev/queue

TradeAC Experiment Queue — Series 2 (Q12+)

Purpose. The next pre-registered batch of experiments, continuing Series 1 (Q01–Q11, exp 33–43, all executed and folded into book/CLAIMS.md / book/EVIDENCE.md). Each entry targets a still-unproven HYPOTHESIS from the book or an open question flagged in CLAIMS.md/book/README.md, and follows the Series-1 discipline: one variable changed vs the exp-26 reference, acceptance fixed BEFORE the run, sequential execution, trace-first, verify-then-close.

Reference / control (MUST reproduce first). exp 26 (21afc6af…, mlflow exp 25) is the campaign baseline; exp 39 (Q07, weekly rebalance) is the best construction. Reference config is byte-reproduced in workflows/exp26/ on the exp/26-… branch and in this dir's workflows/*.yaml.

Config element	exp-26 reference value
Universe	50-ETF panel (UNIVERSE below)
Features	compact stochastic 25-field set (no ou/hmm/moments/garch)
Label	Ref($close,-6)/Ref($close,-1)-1 (5d)
Model	RankICEnsembleLGBModel, seeds 42,7,2026,99,123, lr 0.02, leaves 31, 3000 rounds, ES 200
Segments	train 2016-01-04..2025-09-01 / valid 2025-09-03..2026-01-03 / test 2026-01-04..2026-08-10
Strategy	TopkDropout, topk 10, n_drop 1, risk_degree 0.95
Costs	open 0.0005 / close 0.0015 / min $5, deal $close, SPY benchmark, $1M

Reference metrics to beat (EVIDENCE#015): net_ann +2.13%, net_IR 0.21, gross +7.02%, maxDD −7.69%, RankIC 0.0663, RankICIR 0.2545, L/S Sharpe 4.54. Weekly (Q07, EVIDENCE#028): net +12.51%, IR 1.24, maxDD −4.13%, ~1.1pp cost drag.

The queue (ordered by value × feasibility)

ID	Title / hypothesis	Change vs reference (ONE var)	Acceptance	Config	Ready?
Q12	22d label + weekly recompute — the untested combo: Q05's label edge (IC 0.097, RankIC 0.117) with Q07's cost relief	label → 22d AND strategy → weekly (two coupled, explicitly pre-registered)	net_IR > 0.5, net_ann > +5%, cost drag ≤ 2pp	workflows/q12_label22d_weekly.yaml	✅
Q13	Weekly rebalance reproduction on a 2nd window — Q07 was a single OOS window; reproduce on test 2025-01-02..2025-12-31 before promoting to a live round	segments only (shifted)	net_IR > 0.21, net_ann > +2.13% on the new window	workflows/q13_weekly_second_window.yaml	✅
Q14	Out-of-universe validation — compact stochastic set generalizes off the 50-ETF panel to a single-stock universe	universe → 30 liquid single names	RankIC > 0.03, ICIR > 0.15, net IR > 0 on stocks	workflows/q14_out_of_universe.yaml	⚠️ needs stock-lake backfill (see design)
Q15	5-seed vs single-model clean A/B — seed-count claim (exp 12 idea, re-validated exp 22–24, never a clean A/B)	seeds → 1 (2026)	single-model RankIC/IR < 5-seed ref; net_IR ≥ 0.21 acceptable if ≥ single	workflows/q15_single_seed.yaml	✅
Q16	HMM family added as features — settles "dropping model-specific (ou,hmm) improves signal" (exp 25 tested OU; hmm-as-feature untested)	features += sp_hmm_p_regime1,sp_hmm_state	no improvement: RankIC ≤ 0.0663, net_IR ≤ 0.21	workflows/q16_hmm_features.yaml	✅
Q17	Realized-moments family added — settles "moment/volatility families regress" (exp 11 idea, never clean A/B)	features += sp_rskew_5,sp_rskew_22,sp_rkurt_5,sp_rkurt_22,sp_dsv_5,sp_dsv_22	no improvement: RankIC ≤ 0.0663, net_IR ≤ 0.21	workflows/q17_moments_features.yaml	✅
Q18	OptimalStopControl clean re-test — exp 13/14 claim (TopkDropout > stop-control) never re-tested post-reset	strategy → OptimalStopControl (exp-13 params)	TopkDropout net_IR ≥ stop-control net_IR; document cost drag	workflows/q18_optstop.yaml	✅ (module verified in venv)
Q19	Martingale / variance-ratio study close-out — exp 19 never closed; VR<1 at 5–20d on clean lake	ad-hoc script (no qrun)	VR stats + drift decomposition on 50-ETF panel	designs/q19_martingale_vr.md	✅ script
Q20	Effective independent names (≈4) — eigenvalue analysis on clean-lake covariance	ad-hoc script	eigenvalue spectrum + effective-rank count	designs/q20_effective_names.md	✅ script

Deferred (methodology / infra, P3)

• Purged / walk-forward CV (was queue's old Q12) — methodology, not an alpha lever.
• PSI-based drift-aware retraining cadence — needs a drift-gate module + a retrain decision rule.
• No-trade buffer band / notional-vs-qty sizing — siblings of Q12/Q13; queue only if weekly reproduces.
• Macro/drift overlays (SPY>200d regime gate, momentum tilt) — needs new data pipeline.

Execution protocol (per queued run)

1. Validate the lake first (validate_lake_dataset + rd_status) — clean-lake lesson: silent NaN-drops and hollow coverage invalidate a run. Q14 additionally requires backfilling the single-stock universe (bars + sp/ta features, full range, explicit start/end).
2. Trace before running (rd_trace_start with the hypothesis as rational, fresh experiment_name, evolved_from=auto).
3. Run rd_run_workflow config_path=<abs path to the queue YAML> experiment_name=<fresh name> — wait=false, poll rd_exp_get_run until FINISHED.
4. Verify against acceptance via rd_exp_result (headline + backtest risk).
5. Finish the trace (rd_trace_finish with metrics + evaluation), snapshot any changed contrib modules.
6. Report to the book — PROVE/REFUTE → update book/CLAIMS.md + book/EVIDENCE.md.

Sequential execution only (concurrent runs hang — chat-ideas.md ops lesson). Any custom strategy/module changed here must be copied into the venv site-packages snapshot before rd_run_workflow can import it (see /app/AGENTS.md). As of 2026-08-20 WeeklyRebalanceDropoutStrategy and OptimalStopControl are verified in sync with the venv snapshot; the lake already persists the sp_hmm_* and sp_moments families on the 50-ETF panel.

Provenance

Mined 2026-08-20 from book/CLAIMS.md, book/EVIDENCE.md, book/README.md, book/references/chat-ideas.md, and Series-1 queue/ (Q01–Q11, executed exp 33–43). Reference numbers are post-clean-lake (exp 21+).