Signal scorecard & walk-forward backtest
Does each bucket actually work? Every signal day is re-screened using only data available at that close, then scored on the next session. Primary outcome: enter at next open, exit at next close.
Key finding (2026-05-08 → 2026-11-05, 126 signal days): best historical expectancy was Mean reversion — fade-short leg (after big up day) at 31.2 bp per trade net (hit 55.3%, t = 2.47, n = 358); worst was Short momentum at -23.1 bp (t = -1.88). Statistically meaningful (|t| ≥ 2): Mean reversion — fade-short leg (after big up day) (31.2 bp); Mean reversion (fade, both sides) (21.3 bp).
Historical replay — all picks in each bucket
| Bucket | Trades | Hit rate | Avg win | Avg loss | Payoff | Exp. gross | Exp. net | t-stat | C→C net | O→C ATR | Sharpe net | Liq-adj Sharpe |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Long momentum | 393 | 50.9% | +2.45% | -2.22% | 1.10 | 15.6 bp | 10.6 bp | 0.68 | 32.1 bp | 0.022 | 2.13 | 1.63 |
| Short momentum | 457 | 46.0% | +1.88% | -1.95% | 0.96 | -18.1 bp | -23.1 bp | -1.88 | -31.9 bp | -0.048 | -3.80 | -4.35 |
| Mean reversion (fade, both sides) | 596 | 54.9% | +1.81% | -1.62% | 1.12 | 26.3 bp | 21.3 bp | 2.29 | 14.4 bp | 0.098 | 0.02 | -0.57 |
| Mean reversion — fade-long leg (after big down day) | 238 | 52.5% | +1.63% | -1.56% | 1.04 | 11.5 bp | 6.5 bp | 0.48 | 13.9 bp | 0.037 | 0.68 | 0.23 |
| Mean reversion — fade-short leg (after big up day) | 358 | 56.4% | +1.92% | -1.67% | 1.15 | 36.2 bp | 31.2 bp | 2.47 | 14.7 bp | 0.139 | 0.20 | -0.31 |
| Baseline: every liquid name, long (benchmark) | 15016 | 49.2% | +1.71% | -1.65% | 1.04 | 0.8 bp | -4.2 bp | -2.22 | -0.9 bp | -0.002 | -0.80 | -2.29 |
Historical replay — top 3 ranked picks per day only
| Bucket | Trades | Hit rate | Avg win | Avg loss | Payoff | Exp. gross | Exp. net | t-stat | C→C net | O→C ATR | Sharpe net | Liq-adj Sharpe |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Long momentum | 224 | 50.4% | +2.68% | -2.10% | 1.28 | 31.4 bp | 26.4 bp | 1.27 | 41.9 bp | 0.072 | 2.46 | 1.97 |
| Short momentum | 244 | 45.1% | +1.86% | -1.94% | 0.96 | -21.7 bp | -26.7 bp | -1.60 | -15.2 bp | -0.051 | -3.42 | -3.98 |
| Mean reversion (fade, both sides) | 278 | 51.4% | +1.77% | -1.63% | 1.08 | 12.2 bp | 7.2 bp | 0.53 | 7.0 bp | 0.049 | 0.21 | -0.34 |
| Mean reversion — fade-long leg (after big down day) | 106 | 52.8% | +1.60% | -1.42% | 1.13 | 18.0 bp | 13.0 bp | 0.67 | 28.0 bp | 0.067 | 0.82 | 0.44 |
| Mean reversion — fade-short leg (after big up day) | 172 | 50.6% | +1.87% | -1.76% | 1.06 | 8.7 bp | 3.7 bp | 0.20 | -6.0 bp | 0.038 | 0.53 | 0.09 |
Long momentumShort momentumMean reversion (fade, both sides)Mean reversion — fade-long leg (after big down day)Mean reversion — fade-short leg (after big up day)Baseline: every liquid name, long (benchmark)
Which signals work in which regime?
Same trades split by the regime on the signal day. Cells: net expectancy · hit rate · trades · t-stat. Small n cells are anecdotes, not evidence.
VIX regime (legacy label)
| Bucket | calm: exp. net · hit · n · t | transitional: exp. net · hit · n · t | stress: exp. net · hit · n · t |
|---|---|---|---|
| Long momentum | 7.4 bp · 53% · 73 · 0.2 | 13.0 bp · 50% · 302 · 0.7 | -17.6 bp · 50% · 18 · -0.2 |
| Short momentum | -43.5 bp · 50% · 58 · -1.2 | -30.1 bp · 42% · 326 · -2.1 | 24.2 bp · 49% · 73 · 0.8 |
| Mean reversion (fade, both sides) | 16.6 bp · 63% · 41 · 0.5 | 15.2 bp · 52% · 470 · 1.5 | 57.6 bp · 62% · 85 · 2.1 |
| Mean reversion — fade-long leg (after big down day) | -98.7 bp · 30% · 10 · -2.0 | 2.5 bp · 52% · 183 · 0.2 | 46.1 bp · 56% · 45 · 1.2 |
| Mean reversion — fade-short leg (after big up day) | 53.8 bp · 74% · 31 · 1.2 | 23.3 bp · 51% · 287 · 1.7 | 70.6 bp · 70% · 40 · 1.8 |
Quantitative regime score (regime model)
| Bucket | risk-on: exp. net · hit · n · t | neutral: exp. net · hit · n · t | risk-off: exp. net · hit · n · t |
|---|---|---|---|
| Long momentum | 10.9 bp · 50% · 315 · 0.6 | 9.4 bp · 51% · 78 · 0.3 | — |
| Short momentum | -30.3 bp · 44% · 312 · -2.0 | -32.4 bp · 44% · 109 · -1.4 | 67.4 bp · 50% · 36 · 1.4 |
| Mean reversion (fade, both sides) | 25.6 bp · 54% · 398 · 2.2 | 10.4 bp · 53% · 190 · 0.6 | 68.1 bp · 63% · 8 · 0.6 |
| Mean reversion — fade-long leg (after big down day) | 3.6 bp · 52% · 135 · 0.2 | 5.3 bp · 52% · 95 · 0.2 | 68.1 bp · 63% · 8 · 0.6 |
| Mean reversion — fade-short leg (after big up day) | 36.9 bp · 56% · 263 · 2.5 | 15.5 bp · 55% · 95 · 0.6 | — |
Trend state (SPY ADX)
| Bucket | trending: exp. net · hit · n · t | mixed: exp. net · hit · n · t | choppy: exp. net · hit · n · t |
|---|---|---|---|
| Long momentum | 82.3 bp · 62% · 87 · 2.0 | 1.2 bp · 49% · 112 · 0.1 | -16.2 bp · 46% · 194 · -0.8 |
| Short momentum | -17.1 bp · 46% · 68 · -0.6 | -49.9 bp · 45% · 128 · -2.7 | -11.5 bp · 43% · 261 · -0.6 |
| Mean reversion (fade, both sides) | 8.1 bp · 52% · 143 · 0.4 | -3.6 bp · 52% · 157 · -0.2 | 40.9 bp · 56% · 296 · 2.9 |
| Mean reversion — fade-long leg (after big down day) | 13.9 bp · 53% · 62 · 0.5 | -2.6 bp · 55% · 53 · -0.1 | 6.6 bp · 50% · 123 · 0.3 |
| Mean reversion — fade-short leg (after big up day) | 3.6 bp · 51% · 81 · 0.1 | -4.0 bp · 50% · 104 · -0.2 | 65.3 bp · 61% · 173 · 3.3 |
Live scorecard — picks from stored runs
Actual picks published by the engine (deduplicated per session date, post-close preferred), scored on the following session once it completes.
| Bucket | Trades | Hit rate | Avg win | Avg loss | Payoff | Exp. gross | Exp. net | t-stat | C→C net | O→C ATR | Sharpe net | Liq-adj Sharpe |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Long momentum | 51 | 45.1% | +1.93% | -2.99% | 0.65 | -77.0 bp | -82.0 bp | -1.95 | -88.7 bp | -0.295 | 0.88 | 0.08 |
| Short momentum | 85 | 52.9% | +2.94% | -1.48% | 1.99 | 86.2 bp | 81.2 bp | 2.61 | 53.0 bp | 0.128 | 5.94 | 5.45 |
| Mean reversion (fade, both sides) | 104 | 55.8% | +2.28% | -1.95% | 1.17 | 41.0 bp | 36.0 bp | 1.32 | 32.8 bp | 0.234 | 0.49 | -0.19 |
| Mean reversion — fade-long leg (after big down day) | 48 | 39.6% | +1.40% | -2.17% | 0.65 | -75.4 bp | -80.4 bp | -2.37 | -61.9 bp | -0.140 | -3.52 | -4.09 |
| Mean reversion — fade-short leg (after big up day) | 56 | 69.6% | +2.71% | -1.59% | 1.71 | 140.7 bp | 135.7 bp | 3.69 | 113.9 bp | 0.555 | 2.36 | 1.85 |
Live picks scored on TRAI’s live rules
The same live picks, re-scored the way the live rules would trade them on the next session: enter at the open of the first 5-minute bar at or after 10:00 ET, stop 1×ATR(14), target 2×ATR(14), and exit at 11:00 ET if neither is hit (60-minute time stop). If one 5-minute bar touches both the stop and the target, the stop is assumed. R = net result in ATR units after the same 5 bp + impact costs.
Measurement only — no edge claim until ≥ 30 distinct days scored. Picks from the same day are correlated (one market move hits them all), so judge the per-day mean R and its day-level 90% CI, not the per-trade meanR. The 90% CI column is a bootstrap CI of the per-day mean only (one number per signal day, 2000 day-resamples, fixed seed) — it is not a CI around the per-trade meanR. Those are different estimands: e.g. a per-trade meanR can sit outside the per-day CI without any arithmetic fault (the per-day mean is what the CI covers).
Exit-type mix (live rules)
Target unreachable under the 60-minute window — effective live rule is a time-stop (with a 1×ATR stop). Across all scored exits, TARGET is 0 of 240 (0.0% < 2%). Counts: TIME 235 · STOP 5 · TARGET 0.
| Bucket | Picks | Coverage | Hit rate | Mean R (per-trade) | Median R | Mean net | Stop · Target · Time · No data | Days | Per-day mean R | 90% CI (per-day mean) |
|---|---|---|---|---|---|---|---|---|---|---|
| Long momentum target unreachable — effective rule is time-stop | 51 | 100% | 27.5% | -0.257 | -0.238 | -90.6 bp | 5 · 0 · 46 · 0 | 10 | -0.128 | -0.25 … -0.00 |
| Short momentum target unreachable — effective rule is time-stop | 85 | 100% | 41.2% | -0.038 | -0.066 | 1.8 bp | 0 · 0 · 85 · 0 | 10 | -0.064 | -0.14 … -0.00 |
| Mean reversion (fade, both sides) target unreachable — effective rule is time-stop | 104 | 100% | 53.8% | 0.070 | 0.038 | 16.9 bp | 0 · 0 · 104 · 0 | 10 | 0.050 | -0.03 … 0.13 |
| Mean reversion — fade-long leg target unreachable — effective rule is time-stop | 48 | 100% | 54.2% | 0.020 | 0.026 | 6.4 bp | 0 · 0 · 48 · 0 | 9 | 0.125 | -0.04 … 0.37 |
| Mean reversion — fade-short leg target unreachable — effective rule is time-stop | 56 | 100% | 53.6% | 0.113 | 0.067 | 25.9 bp | 0 · 0 · 56 · 0 | 8 | 0.044 | -0.04 … 0.14 |
| All live picks target unreachable — effective rule is time-stop | 240 | 100% | 43.8% | -0.037 | -0.042 | -11.3 bp | 5 · 0 · 235 · 0 | 11 | -0.043 | -0.12 … 0.02 |
Assumptions & limits.
- Costs: 5 bp round trip on every trade. Liquidity-adjusted Sharpe also subtracts round-trip impact = 2 × 0.5 × σ_daily × √($1,000,000 / ADV20).
- Primary outcome: enter at next session open, exit at next session close (open→close). Secondary: signal close → next close. Both are open→close PROXIES, not TRAI's live rule set (no entry 9:30–10:00, 1×ATR stop, 2×ATR target, 60-min time stop); live picks are also scored on those rules in `liveRules` (measurement only).
- Historical replay uses daily bars only: the live "close vs VWAP" condition is approximated by close vs daily typical price (H+L+C)/3.
- Replay uses today’s universe list (survivorship bias: names were selected with hindsight).
- Sharpe uses the equal-weight portfolio of each day's picks (0 on days with no picks), annualized with √252. Expectancy/hit-rate are per trade.
- Replay window ~6 months: a single market environment. Treat results as hypotheses to monitor, not proof. Raw trades: /trai/api/scorecard (or ?lite=1 for summaries).