book: scaffold + ch00 (execution trail as spine) — evidence exp 8-31, round 3
This commit is contained in:
TradeAC Book Agent
@@ -0,0 +1,325 @@
|
||||
---
|
||||
name: tac-algo-trade
|
||||
description: Guide agents to run the TradeAC scheduled algo-trading flow end-to-end. First backfill the data lake for all symbols up to the latest completed trading day, then — given a reference MLflow run (experiment_name + run_id) — re-train the same model configuration on a rolling window (4 years up to the latest completed trading day), generate fresh signals, run the selected strategy into a target order list, and place the orders on the Alpaca paper account — chaining every step from the previous one's output. Uses the `tac-engine` lake MCP tools (get_lake_bars, backfill_lake_calendar, get_lake_coverage) for data, the `tac-qlib-rd` MCP tools (rd_train, rd_predict, rd_strategy_targets, rd_exp_*) for the quant side, and the `tac-engine` MCP tools (place_order, list_orders, list_positions, get_news, ...) for execution.
|
||||
---
|
||||
|
||||
# tac-algo-trade
|
||||
|
||||
Scheduled algo trading for the TradeAC paper account. Each scheduled execution (1) backfills the lake so it is current through the latest completed trading day, (2) re-trains the reference run's configuration on the most recent 4 years of lake data, (3) predicts, (4) derives an order list from the strategy, and (5) places orders on Alpaca.
|
||||
|
||||
This is the skill the app's scheduler invokes (`/dashboard/scheduler`). It chains strict step-to-step outputs: **do not skip ahead, do not fabricate outputs — every step consumes the artifact path returned by the previous one.**
|
||||
|
||||
## MCP tools
|
||||
|
||||
- Data side: `tac-engine` lake tools (see `tac-engine/skills/tradeac-lake/SKILL.md`) — `get_lake_coverage`, `backfill_lake_calendar`, `get_lake_bars` (lazy backfill), `get_lake_status`.
|
||||
- Quant side: `tac-qlib-rd` (see `tradeac-rd/SKILL.md`) — `rd_status`, `rd_exp_get_experiment`, `rd_exp_input`, `rd_train`, `rd_predict`, `rd_strategy_targets`, `rd_exp_get_run`.
|
||||
- Execution side: `tac-engine` (see `tac-engine/skills/tradeac-alpaca/SKILL.md`) — `get_account`, `list_positions`, `list_orders`, `place_order`, `get_stock_snapshot`, `get_stock_latest_quotes`, `get_news`.
|
||||
- Round book: `tac-rd-book` — the execution trail (see the "Round book" section below). `round_create` / `round_update`, `fact_record`, `intent_set`, `decision_record`, `round_sync_fills`, `round_update_status`, `book_reconcile`, `trail_funnel`.
|
||||
|
||||
All steps use the MCP tools directly. Never hand-compute scores, read mlruns files directly (`mlruns.db` / pickles — the store is Postgres via `DATABASE_URL` when set), or script the MCP servers yourself. This is a **paper** account — trade normally, size each order per the strategy's target weight × live equity (whole shares), capped by available buying power, and skip anything untradeable.
|
||||
|
||||
## Inputs
|
||||
|
||||
- `experiment_name` — MLflow experiment of the reference run.
|
||||
- `run_id` — the reference run inside that experiment (its saved `config` artifact is the source of truth for the whole pipeline).
|
||||
- `strategy` (optional) — a workflow YAML from `tac-qlib/workflows/*.yaml` defining the strategy sizing (e.g. `topk` / `n_drop` / `risk_degree`, benchmark, costs). Default: the reference run's own backtest config.
|
||||
- Time context: today's date in the scheduling city's timezone.
|
||||
|
||||
## Step 1 — Backfill the lake (data currency)
|
||||
|
||||
The retrain must see every symbol up to the latest available bar — do **not** train on stale data.
|
||||
|
||||
1. `get_lake_coverage` `{"market":"US","timeframe":"1d"}` → read each symbol's loaded window; the **last loaded date** across the universe is your backfill start.
|
||||
2. `backfill_lake_calendar` `{"market":"US","symbols":"all","start":<backfill start>,"end":<today>}` → seed the trading-day set first so the 1d completeness check knows which days to expect.
|
||||
3. `get_lake_bars` `{"market":"US","symbols":"all","timeframe":"1d","start":<backfill start>,"end":<today>,"lazy":true,"quiet":true}` → backfill every symbol's gap (Alpaca historical bars) and persist to the lake. Use `quiet: true` so the tool returns a per-symbol `{count, first_t, last_t}` summary instead of echoing back thousands of bar rows. Alpaca has no bar for today until the session closes, so the latest bar landed is the **latest completed trading day** `D` (for a Monday run this is Friday).
|
||||
|
||||
Confirm with `rd_status` (calendar range + coverage) that the lake is populated through `D`. **Output: `D`, the latest completed trading day.**
|
||||
|
||||
## Step 2 — Inspect the reference run
|
||||
|
||||
`rd_exp_get_experiment` with `experiment_id` (or `rd_exp_input` with `run_id`) → extract from the run's `config` artifact:
|
||||
|
||||
- handler config: `universe` (instruments), `features`, `label`, `freq`
|
||||
- model kwargs: `learning_rate`, `num_leaves`, `n_estimators`, `colsample_bytree`, `subsample`, `subsample_freq`, `reg_alpha`, `reg_lambda`, `seed`
|
||||
- strategy sizing: `topk` / `n_drop` / `risk_degree`, costs, benchmark
|
||||
|
||||
Record these — they define the retrain. **Output: config values above.**
|
||||
|
||||
## Step 3 — Open the traced experiment (git lineage)
|
||||
|
||||
Every scheduled run is a **traced experiment** on the tac-qlib-custom lineage: a row in the
|
||||
`rd_experiments` table plus a per-experiment git branch in the `experiments` submodule,
|
||||
forked from the predecessor's branch. **This is part of the run — do it automatically, do
|
||||
not wait for the user to prompt** (see `tac-qlib/skills/tac-qlib-custom/SKILL.md`,
|
||||
"Experiment traceability", for the full procedure and env vars).
|
||||
|
||||
1. Resolve the predecessor: if the reference run (`experiment_name` / `run_id` from Step 2)
|
||||
is itself traced, reuse its traced id as `evolved_from`; otherwise use `--evolved-from auto`
|
||||
(semantic search over existing rationals).
|
||||
2. Open the trace — this inserts the row, forks the branch from the predecessor and pushes it (via the `rd_trace_*` MCP tools on tac-qlib-rd):
|
||||
|
||||
```
|
||||
rd_trace_init
|
||||
rd_trace_start rational="scheduled algo retrain on <D>: <ref exp>/<ref run> re-trained on 4y -> live paper orders" \
|
||||
details="<universe / features / label / model / strategy sizing from the reference run config>" \
|
||||
experiment_name=<THE RUN'S experiment name — see naming below> \
|
||||
evolved_from=<predecessor id or auto> \
|
||||
session_id="<this chat's opencode session id>"
|
||||
# -> {"experiment_id": N, "branch": "...", "evolved_from": ..., "base_branch": ...}
|
||||
```
|
||||
|
||||
**Experiment naming (unique per run):** every retrain runs into its OWN
|
||||
experiment — `<reference experiment name>-<epoch seconds>` (e.g.
|
||||
`tac-basic-short-1786883261`). The scheduler prompt names the exact
|
||||
experiment for you; use that name for `rd_trace_start experiment_name`,
|
||||
`rd_train`'s `experiment_name`, and the round's `experiment_name`. **Never**
|
||||
reuse the reference experiment name for this run's trace node — reusing it
|
||||
creates duplicate lineage entries with the same name and a wrong parent
|
||||
chain (seen with `tac-basic-short`).
|
||||
|
||||
The tool returns `experiment_id` / `branch` as JSON — record them; every
|
||||
later `rd_trace_*` call uses the id. Commit the run's files (workflow YAML /
|
||||
notes) with `rd_trace_commit experiment_id=<N> message="..."` as you go.
|
||||
|
||||
3. **Round window** — the execution trail for `D`:
|
||||
- **If the scheduler pre-created it** (your instructions name a `ROUND_ID` / `target_date` / `source`) — **skip `round_create`** and use that `ROUND_ID`. If the `D` you computed in Step 1 differs from the given `target_date`, correct it first with `round_update {round_id:<ROUND_ID>, target_date:<D>}` (weekday rule can't see NYSE holidays; the agent reconciles).
|
||||
- Otherwise create it yourself (idempotent: a second scheduled run for the same day reuses the open window):
|
||||
|
||||
```
|
||||
round_create {target_date:<D>, signal_date:<D>, source:"scheduled",
|
||||
rd_experiment_id:<EXPERIMENT_ID>, experiment_name:<the run's unique experiment name>}
|
||||
# -> round_id (record it; every round-book call below uses it)
|
||||
```
|
||||
|
||||
**Output: `EXPERIMENT_ID` (and its branch), `ROUND_ID`.**
|
||||
|
||||
## Step 4 — Re-train with the rolling window
|
||||
|
||||
Call `rd_train` with the **exact same configuration** from Step 2, only the dates change:
|
||||
|
||||
- `train_start` = 4 years before `D` (same day-of-month), `train_end` = `D`
|
||||
- **Validation is optional** — qlib supports omitting it, so omit `valid_start`/`valid_end`/`test_start`/`test_end` (pass them empty). If the tool/your run requires a holdout for sanity, use a short recent `valid` window only; never reserve data the live model needs.
|
||||
- `record_analysis=false` (we only need the model; no SignalRecord/PortAnaRecord on a holdout we don't use)
|
||||
- `wait=false` (recommended) — `rd_train` returns immediately and the fit runs in the background; poll `rd_exp_get_run` (or `rd_exp_list` filtered to the experiment) until the newest run's status is `FINISHED`, then take its `run_id`. With `wait=true` the call blocks until the fit completes — fine when the window is small, but a 4y LightGBM fit can outlive the MCP call timeout, which forced manual recovery in an earlier run.
|
||||
- `out_dir` — the working directory for this run (e.g. `tac-algo-output`)
|
||||
- `experiment_name` — the **run's unique experiment name** (the scheduler prompt names it: `<reference experiment name>-<epoch seconds>`). This is the SAME name used for `rd_trace_start experiment_name` and the round's `experiment_name`. Do not reuse the reference experiment name.
|
||||
|
||||
Keep the same `universe`, `features`, `label`, and every model hyper-parameter. **Output: the new run's `model_path` (and its `run_id`).**
|
||||
|
||||
> If a 4-year window is slower than the schedule allows, use the largest trailing window you can complete and say so in the summary — never silently shrink the horizon.
|
||||
|
||||
Pin the new training run to the round window:
|
||||
|
||||
```
|
||||
round_update {round_id:<ROUND_ID>, run_id:<new run_id>, model_path:<params.pkl path>}
|
||||
```
|
||||
|
||||
## Step 5 — Generate predictions (the signal)
|
||||
|
||||
Call `rd_predict` with `model_path` = the path returned by Step 4 (preferred over `run_id` since it is the freshly-trained artifact):
|
||||
|
||||
- `test_start` = `D`, `test_end` = `D` (the just-completed trading day — this is the signal we trade on)
|
||||
- same `universe` / `features` / `label` as Step 2
|
||||
- `out_dir` = the same working directory
|
||||
|
||||
**Output: `pred_path` (pred.pkl) and the score ranking.** The model's predicted score per instrument IS the alpha signal for day `D` — top-scored names are candidates.
|
||||
|
||||
Record the signal into the round book (one `fact_record` per top-scored name, plus the strategy config and the market snapshot at prediction time):
|
||||
|
||||
```
|
||||
fact_record {round_id:<ROUND_ID>, kind:"signal_score", symbol:<ticker>, payload:{"pred":<score>, "rank":<rank>}, source:"rd_predict"}
|
||||
fact_record {round_id:<ROUND_ID>, kind:"strategy_config", payload:{...strategy sizing...}, source:"reference config"}
|
||||
fact_record {round_id:<ROUND_ID>, kind:"market_snapshot", payload:{<ticker>: {last:<px>, change_pct:<%>, vol:<vol>, updated:<ts>}, ...}, source:"get_stock_snapshots / get_stock_latest_quotes"}
|
||||
```
|
||||
|
||||
`market_snapshot` freezes the market state **when the prediction was made** — the latest price / % change / volume per universe name, so the signal can later be judged against what the market looked like at that moment.
|
||||
|
||||
## Step 6 — Run the configured strategy, derive the target order list
|
||||
|
||||
**First pull the current portfolio — it is an input to the strategy step** (the order list is a delta, not a full rebuild):
|
||||
|
||||
- `get_account` → cash / buying power **and total equity** (equity sizes the positions; buying power caps total buys)
|
||||
- `list_positions` → current holdings and their market value
|
||||
|
||||
Then run the strategy **exactly as it was configured in the reference run** — this works for any model/strategy, not just TopkDropout. The reference run's saved `config` artifact (from `rd_exp_input`, Step 2) carries the strategy configuration from its backtest/record block (e.g. `TopkDropoutStrategy` kwargs: `topk`, `n_drop`, `risk_degree`, or any custom strategy's own kwargs, plus costs, `account`, `benchmark`). **Use those values — not tool defaults.** The model's score is the signal the strategy consumes; the strategy's config decides allocation.
|
||||
|
||||
Call `rd_strategy_targets` with:
|
||||
|
||||
- `pred_path` = the signal from Step 5
|
||||
- the run-configured `topk` / `n_drop` / `risk_degree` (from the reference run config)
|
||||
- `account` = the **live account equity** from `get_account` (a new account is not a $1M book — sizing against `$1M` when equity is far smaller produces oversized orders)
|
||||
- `prices` = a JSON `{symbol: price}` of latest quotes (from `get_stock_latest_quotes`) so the tool floors each order to whole shares (`qty`) and reports `expected_price` / `invested`
|
||||
- `risk_limits` = the round's risk-limit spec JSON (see below) — the SAME spec that `rd_backtest` uses, so live gating is provable against backtest
|
||||
- `equity` / `peak_equity` = live equity and its trailing peak (from `get_portfolio_history`) when `risk_limits.drawdown_pause_pct` is set
|
||||
|
||||
The tool applies the exact TopkDropout selection on day `D`: rank the cross-sectional scores, **drop the top `n_drop`**, take the next `topk` as buys, sized at `account × risk_degree / topk` per name. It then applies `risk_limits` as pre-gates — liquidity floor (drops names with avg daily dollar volume below `liquidity_floor_adv`), per-name `size_cap_pct` of equity, `concentration_cap_pct` of equity on total deployed, and `drawdown_pause_pct` (equity ≤ (1−pause)×peak ⇒ no buys). **Output: the deterministic target buy list** (`symbol`, `rank`, `score`, `side`, `notional`, `qty`), the full `ranking`, and `risk_limits_applied` (which limits cut what — record it). **No manual strategy replication** (an earlier run's hand-rolled sizing silently dropped the n_drop and bought the wrong names).
|
||||
|
||||
> If the strategy in the run/workflow config does not fit TopkDropout's `topk`/`n_drop`/`risk_degree`, apply the strategy's own rules to the Step 5 scores directly to derive the target portfolio, still bounded by `get_account` buying power and today's `list_positions`.
|
||||
|
||||
Then convert the target portfolio into an order list against the current holdings:
|
||||
|
||||
- For each target ticker compute the **delta** vs. what the account already holds: buy the shortfall, sell the excess. Do not blindly re-buy names already held, and do not sell names that are not in the portfolio.
|
||||
- **Fresh account (no positions):** the target portfolio is entirely new buys — emit no sell orders, and size each buy from the tool's `qty` (or `notional` ÷ latest quote), capped by buying power.
|
||||
- Skip any ticker whose delta is ~0 (already at target) so you don't churn held names.
|
||||
- Cap total buy size to available buying power. Drop any ticker with no score in Step 5 or no tradable quote.
|
||||
|
||||
**Output: the explicit order list** (ticker, side, qty, order type).
|
||||
|
||||
**Write the target into the round book** — this is the intent the round reconciles against (versions auto-increment; a second strategy pass for the same round supersedes the first):
|
||||
|
||||
```
|
||||
fact_record {round_id:<ROUND_ID>, kind:"account_state", payload:{"equity":<live equity>, "buying_power":<bp>}, source:"get_account"}
|
||||
fact_record {round_id:<ROUND_ID>, kind:"position_state", symbol:<ticker>, payload:{"shares":<held>}, source:"list_positions"}
|
||||
fact_record {round_id:<ROUND_ID>, kind:"risk_check", payload:{"risk_limits":{...spec...}, "applied":{...risk_limits_applied from the tool...}, "equity":<equity>, "peak_equity":<peak>}, source:"rd_strategy_targets"}
|
||||
round_update {round_id:<ROUND_ID>, account_equity_at_sizing:<live equity>, strategy_snapshot:{topk, n_drop, risk_degree, costs, benchmark, risk_limits:{liquidity_floor_adv?, size_cap_pct?, concentration_cap_pct?, drawdown_pause_pct?}}}
|
||||
intent_set {round_id:<ROUND_ID>, target_portfolio:[{symbol, side, qty, notional, expected_price, score, rank}...],
|
||||
raw_strategy_output:{...the strategy output as computed...}, reason:"topk<N> from <ref run>"}
|
||||
```
|
||||
|
||||
**Risk-limit spec (B)**: the round's `risk_limits` (a JSON map with any of `liquidity_floor_adv`, `size_cap_pct`, `concentration_cap_pct`, `drawdown_pause_pct`) is the single source of truth — **the same spec is passed to `rd_backtest` when calibrating** (B2), folded into `rd_train`'s PortAnaRecord via `risk_degree`, and consulted by `rd_strategy_targets` live. Store it verbatim in `strategy_snapshot.risk_limits`. When the tool's `risk_limits_applied` reports a limit that cut targets (dropped liquidity / capped sizing / drawdown pause), record it — the audit trail proves the limit fired live exactly as the calibration predicted. If `drawdown_pause_pct` fired and produced an empty target list, **settle the round as open→settled with no orders** rather than forcing buys (that is the intended behavior).
|
||||
|
||||
**Record the evidence behind each selected name** — the feature snapshot and the decision rationale, so the fact table can answer *why this symbol was ranked top-K*:
|
||||
|
||||
- `symbol_features` — the model-input feature values that produced the score on day `D` (the top features by `rd_exp_model` importance, plus the handful most relevant for that name — e.g. trend slopes, RSI, volume/vol ratios, MACD):
|
||||
```
|
||||
get_lake_ta {symbol:<ticker>, timeframe:"1d", start:<~60d before D>, end:<D>, persist:true, quiet:true} # (re)compute TA + sp_* columns up to D
|
||||
get_lake_features {symbol:<ticker>, timeframe:"1d", start:<D>, end:<D>} # read the D row; if 0 rows, the persisted features are stale -> persist first as above
|
||||
rd_exp_model {run_id:<new training run_id>, tree_id:0, max_depth:4} # feature_importances + tree nodes
|
||||
fact_record {round_id:<ROUND_ID>, kind:"symbol_features", symbol:<ticker>,
|
||||
payload:{"score":<score>, "rank":<rank>, "features":{<top feature>:<value>, ...}}, source:"get_lake_features / rd_exp_model"}
|
||||
```
|
||||
`get_lake_features` returns 0 rows for day `D` when the persisted feature files were last written before `D` (they are per-symbol parquet files that only extend to the last time they were computed). In that case **first persist** with `get_lake_ta ... persist:true` (and `get_lake_sp` when the model uses `sp_*` columns — the rd_train feature list from Step 2 tells you which), then read `get_lake_features` for `D` again — it must return a row per ticker.
|
||||
- `decision_justification` — **concise** (under 500 words total, aim for 2–4 sentences per name): why the model ranked the symbol top-K. Ground it in the actual data — the `rd_exp_model` tree path (which feature conditions led the row down the high-score branch) and the `symbol_features` values — not generic commentary:
|
||||
```
|
||||
fact_record {round_id:<ROUND_ID>, kind:"decision_justification", symbol:<ticker>,
|
||||
payload:{"score":<score>, "rank":<rank>, "why": "<2-4 sentences, e.g. 'strong 5d trend slope + rising volume ratio put TSLA above $sp_trend_slope_60 threshold, sending it down the high-score branch (leaf value +0.0545); RSI recovering but not overbought.'>"},
|
||||
source:"rd_exp_model tree + feature snapshot"}
|
||||
```
|
||||
|
||||
## Step 7 — Execution context + news sentiment gate
|
||||
|
||||
Before placing anything, per candidate ticker:
|
||||
|
||||
1. `get_account` (buying power), `list_orders` (open orders), `list_positions` (current holdings).
|
||||
2. `get_stock_snapshot` / `get_stock_latest_quotes` → sanity-check each quote: skip tickers with no quote, a stale/illiquid quote (wide spread or near-zero volume), or a halt. Use the latest quote, not just the model score, for sizing and order type.
|
||||
3. `get_news` with `symbols=<ticker>`, `limit=20`, `include_content=true` → assign a sentiment score **−3 (strongly negative) … +3 (strongly positive)**.
|
||||
|
||||
**Sentiment gate:** if sentiment strongly contradicts the signal — a **BUY** with sentiment ≤ −2 or a **SELL** with sentiment ≥ +2 — **cancel** that order and record it as `cancelled: sentiment conflict`. Tickers with no news or neutral sentiment (−1..+1) trade normally.
|
||||
|
||||
Record the evidence per candidate into the round book (so the reconcile step can explain every skip):
|
||||
|
||||
```
|
||||
fact_record {round_id:<ROUND_ID>, kind:"quote", symbol:<ticker>, payload:{bid, ask, last, spread_bps}, source:"get_stock_snapshot"}
|
||||
fact_record {round_id:<ROUND_ID>, kind:"news_sentiment", symbol:<ticker>, payload:{"sentiment":<−3..+3>, "headline":<top headline>}, source:"get_news"}
|
||||
```
|
||||
|
||||
## Step 8 — Place orders on Alpaca
|
||||
|
||||
For each surviving order in the Step 6 list (respecting the gate): call the `tac-engine` `place_order` tool with the ticker, side, qty and order type. Then verify with `list_orders` / `list_positions` that the intended changes went through.
|
||||
|
||||
**Record every decision in the round book** — placed orders AND deliberate skips, each with its reason (this is what the reconcile / funnel view reads):
|
||||
|
||||
```
|
||||
# each placed order (order id from the place_order response):
|
||||
decision_record {round_id:<ROUND_ID>, symbol:<ticker>, side:<buy|sell>, qty:<qty>, order_type:<type>,
|
||||
expected_price:<last quote px>, status:"placed", reason:"placed",
|
||||
intent_id:<intent id from intent_set>, alpaca_order_id:<alpaca order id>, client_order_id:<cl id>}
|
||||
# each gate cancel / skip (delta≈0, no quote, illiquid, halt, bp cap, sentiment conflict, no score, risk limit):
|
||||
decision_record {round_id:<ROUND_ID>, symbol:<ticker>, side:<side>, qty:<qty>, status:"skipped",
|
||||
reason:"sentiment_conflict"|"illiquid"|"no_quote"|"halt"|"delta_zero"|"bp_cap"|"no_score"|"risk_limit",
|
||||
reason_detail:<short why>, intent_id:<intent id>}
|
||||
```
|
||||
|
||||
**Sync fills** — pull Alpaca's order state into the round (pass the `list_orders` output as `orders` so no API call is needed; unmatched orders are reported back):
|
||||
|
||||
```
|
||||
round_sync_fills {round_id:<ROUND_ID>, orders:[{id, client_order_id, symbol, side, qty, filled_qty, filled_avg_price, status}...]}
|
||||
```
|
||||
|
||||
## Step 9 — Evidence check, close the traced experiment + summarize
|
||||
|
||||
**Evidence gate — run this BEFORE committing/closing. Do not skip, do not "summarize only".** Query the round and confirm every evidence kind is present; record anything missing right now, then re-query:
|
||||
|
||||
```
|
||||
fact_query {round_id:<ROUND_ID>} # or per-kind: fact_query {round_id:<ROUND_ID>, kind:"<kind>"}
|
||||
```
|
||||
|
||||
For each of the per-universe kinds (`signal_score`, `market_snapshot`, `quote`, `news_sentiment`, `symbol_features`, `decision_justification`) count that you recorded one per symbol you processed; `strategy_config`, `account_state`, `position_state` once each. If any kind is missing or any target symbol is missing from a kind, **go back and `fact_record` it now** (use the persist→read recipe in Step 6 for `symbol_features`). Only when every kind above is present, proceed:
|
||||
|
||||
1. Commit the run artifacts to the experiment branch: `rd_trace_commit experiment_id=<EXPERIMENT_ID> message="algo run <D>: orders placed"`.
|
||||
2. Close the lineage — re-embeds the rational/details, records metrics/evaluation, commits + pushes:
|
||||
```
|
||||
rd_trace_finish experiment_id=<EXPERIMENT_ID> \
|
||||
ref_id=<new training run_id from Step 4> \
|
||||
evaluation="<outcome of today's trade: target vs placed, cancellations>" \
|
||||
metrics='{"n_buys":N,"n_sells":M,"n_cancelled":K}' \
|
||||
mlruns_dir=<lake>/mlruns/<exp_id>/<run_id>
|
||||
```
|
||||
3. **Settle the round** — reconcile and close the window:
|
||||
```
|
||||
book_reconcile {round_id:<ROUND_ID>} # residual vs target, per-symbol reasons
|
||||
trail_funnel {round_id:<ROUND_ID>} # targets -> decided -> placed -> filled, skips by reason
|
||||
round_update_status {round_id:<ROUND_ID>, status:"settled", summary_metrics:{...funnel + invested...}}
|
||||
```
|
||||
4. **Close the loop** — record the round's execution economics for the next run's tuning:
|
||||
- `round_metrics` → the round's invested notional, turnover, slippage bps, estimated cost, cost-as-% of gross (the `fetchPriorRoundFeedback` in the scheduler injects these into the NEXT run's prompt automatically).
|
||||
- If the round had fills, run `rd_factor_attribution` over the round window (pass the `get_portfolio_history` equity curve as `portfolio_equity`, benchmark e.g. `IVV`, realized slippage+cost bps from `round_metrics`, expected values from the calibration) and record the result:
|
||||
```
|
||||
fact_record {round_id:<ROUND_ID>, kind:"attribution", payload:{beta, alpha_annualized_pct, pnl_beta, pnl_alpha, drift_alarm}, source:"rd_factor_attribution"}
|
||||
```
|
||||
- A `drift_alarm` in the attribution means live execution cost is deviating from the backtest assumption — re-run `rd_risk_calibrate` before the next round and tighten sizing/limits.
|
||||
5. End your reply with the compact summary: date `D`, reference run (`experiment_name` / `run_id`), new training run (`run_id` / `model_path`), window (4y → `D`), number of scores, top names, per-ticker sentiment scores, what was bought/sold, which orders were cancelled by the sentiment gate (and why), and any skipped trades (with reasons).
|
||||
|
||||
## Example
|
||||
|
||||
```
|
||||
experiment_name=tac-rd run_id=<ref-uuid> strategy=tune_run1_wider_5d.yaml
|
||||
1. get_lake_coverage {US,1d} -> last loaded date; backfill_lake_calendar; get_lake_bars lazy -> lake current -> D
|
||||
2. rd_exp_input run_id=<ref-uuid> -> universe=all, features=KR..(ta fields), label=Ref($close,-2)/Ref($close,-1)-1, lr=0.05, leaves=15 ... topk/n_drop from the run's backtest config
|
||||
3. EXP_NEW=<ref exp>-<epoch seconds> # unique per run (scheduler names it)
|
||||
rd_trace_init && rd_trace_start experiment_name=$EXP_NEW evolved_from=auto -> experiment_id / branch
|
||||
# scheduler usually pre-creates the round (ROUND_ID in the instructions) -> skip round_create, use it
|
||||
round_create {target_date:<D>, signal_date:<D>, source:"scheduled", rd_experiment_id:<EXPERIMENT_ID>, experiment_name:$EXP_NEW} -> ROUND_ID
|
||||
4. rd_train experiment_name=$EXP_NEW train_start=<D-4y> train_end=<D> record_analysis=false wait=false out_dir=tac-algo-output
|
||||
# -> returns immediately; poll rd_exp_get_run until status FINISHED -> run_id <new-uuid>, model_path tac-algo-output/params.pkl
|
||||
round_update {round_id:<ROUND_ID>, run_id:<new-uuid>, model_path:"tac-algo-output/params.pkl"}
|
||||
5. rd_predict model_path=tac-algo-output/params.pkl test_start=<D> test_end=<D>
|
||||
# -> pred_path tac-algo-output/pred.pkl, score head ...
|
||||
fact_record {kind:"signal_score", symbol:<ticker>, payload:{pred, rank}} per top name
|
||||
6. get_account + list_positions # current portfolio as strategy input; account=live equity
|
||||
get_stock_latest_quotes -> prices JSON for sizing
|
||||
rd_strategy_targets pred_path=tac-algo-output/pred.pkl signal_date=<D> \
|
||||
topk=<from run config> n_drop=<from run config> risk_degree=<from run config> account=<live equity> prices='{...}' \
|
||||
risk_limits='{"liquidity_floor_adv":5000000,"size_cap_pct":8,"concentration_cap_pct":30}' equity=<equity> peak_equity=<peak>
|
||||
# -> deterministic target buys (symbol/rank/score/notional/qty); delta vs list_positions -> order list (fresh account = all buys)
|
||||
fact_record {kind:"risk_check", payload:{risk_limits:{...}, applied:{...risk_limits_applied...}, equity, peak_equity}}
|
||||
round_update {round_id:<ROUND_ID>, account_equity_at_sizing:<equity>, strategy_snapshot:{topk, n_drop, risk_degree, costs, benchmark, risk_limits:{...}}}
|
||||
intent_set {round_id:<ROUND_ID>, target_portfolio:[{symbol, side, qty, expected_price, score, rank}]} -> intent_id
|
||||
7. get_news per ticker -> sentiment gate; fact_record quote + news_sentiment per ticker
|
||||
8. place_order ... per surviving delta; decision_record per placed + skipped (with reason)
|
||||
round_sync_fills {round_id:<ROUND_ID>, orders:[...list_orders output...]}
|
||||
9. rd_trace_commit experiment_id=<EXPERIMENT_ID> + rd_trace_finish experiment_id=<EXPERIMENT_ID> ref_id=<new-uuid>
|
||||
book_reconcile + trail_funnel; round_update_status {status:"settled", summary_metrics:{...}}; summary
|
||||
```
|
||||
|
||||
## Round book — the execution trail
|
||||
|
||||
Every scheduled run writes its decision→fill trail to Postgres via the `tac-rd-book`
|
||||
tools, mirroring the `/dashboard/rounds` UI. The round is the link between the scheduler
|
||||
run, the traced experiment, and the actual account activity:
|
||||
|
||||
```
|
||||
scheduler_runs ──► ROUND ──► rd_experiments
|
||||
│ fact_events evidence: signal_score / market_snapshot / quote / news_sentiment / account_state / position_state / symbol_features / decision_justification / risk_check
|
||||
│ round_intents versioned target portfolios (new version supersedes old)
|
||||
│ round_decisions per-symbol: placed OR skipped, each with a reason (incl. risk_limit)
|
||||
└──► round_orders execution rows (Alpaca order id + fills), synced via round_sync_fills
|
||||
```
|
||||
|
||||
`book_reconcile` returns the per-symbol residual (target qty − filled qty, with the reason
|
||||
it did not fill) plus cash/BP impact, slippage bps and estimated cost — that is the answer
|
||||
to "why is the account not at the target portfolio". `round_metrics` reports the round
|
||||
roll-ups (invested notional, turnover, slippage bps, estimated cost, cost-as-% of gross);
|
||||
`trail_funnel` gives the counts (targets → decided → placed → filled, skips by reason).
|
||||
All surface unchanged in the UI. When a round fires `drawdown_pause_pct`, its `round_metrics`
|
||||
will show `invested_notional: 0` — that is the pause working, not a broken round.
|
||||
Reference in New Issue
Block a user