# VCET-at-Entry Validation Plan (Phase 4)

**Date:** 2026-06-04
**Author:** Claude Code (plan-only; no execution)
**Status:** PLAN-ONLY. No DB writes, no VCET code edits, no Phase 4 execution. Awaits explicit approval before any step lands.
**Sister docs:** [CANONICAL_STRATEGY_PERFORMANCE_REPORT.html](./CANONICAL_STRATEGY_PERFORMANCE_REPORT.html) · [VWAP_PULL_IN_QUALITY_GATE.md](./VWAP_PULL_IN_QUALITY_GATE.md) · [VWAP_PULL_IN_SPLIT_DIAGNOSTIC.md](./VWAP_PULL_IN_SPLIT_DIAGNOSTIC.md)

---

## 0. Goal & hypothesis

**Goal:** test whether VCET states empirically separate the Pull-In split. Concretely:

| Hypothesis | Expected pattern if VCET works |
|---|---|
| **H1 — Converters live in expansion** | ≥70% of `vwap-pull-in -> trend` winners (14 trades) entered during `EARLY_EXPANSION`, `EXPANSION_CONFIRMED`, or `TREND_UP/DOWN` |
| **H2 — Leakers live in indecision** | ≥60% of plain `vwap-pull-in` losers (80 trades) entered during `ROTATION`, `FAILED_EXPANSION_TENTATIVE/CONFIRMED`, or weak `COMPRESSION` (score < 50) |
| **H3 — Band geometry predicts** | At entry: PI→Trend wins have higher `vwap_band_width_pct` (bands fanned) and steeper `vwap_slope`; plain PI losses have flatter slope + narrower bands |
| **H4 — Chase / Extended sentinel** | The 56 `chase-extended` trades (currently net +$200, 57% WR — surprising) entered in `EXPANSION_CONFIRMED` extended territory. Validates the "extended chase that worked because momentum was real" read |

**Success criterion for this plan to be approved for execution:** all four hypotheses are testable on a usable sample size (n ≥ 30 per group). If sample availability fails this (per §7 risks), the plan adjusts scope before execution.

**Failure mode worth naming:** if VCET states are statistically indifferent between Pull-In winners and losers, that argues VCET v1.0 is the wrong layer for this discriminator and RVOL+gap (the existing Quality Gate) is the better filter — no harm done, the gate still works.

---

## 1. Data needed

Listed with availability status. Items marked ⚠ are the real blockers (see §7).

### 1.1 Trade-side (have it — all on `main.trade_context`)

| Field | Source | Status |
|---|---|---|
| `canonical_strategy_slug` | `main.trade_context` | ✅ populated for 310 / 310 trades post-Phase-2 |
| `entry_time` | `main.trade_context` (naive TIMESTAMP, ET local per content) | ✅ |
| `symbol` | `main.trade_context` | ✅ |
| `net_pnl`, `pnl_per_share`, `side` | `main.trade_context` | ✅ |
| `exit_time` (for state-after-entry) | `main.trade_context` | ✅ (where populated) |
| `raw_classifier_output` | `main.trade_context` | ✅ post-Phase-1.6 |
| `mtf_grand_total`, RVOL, gap%, ATR | `main.trade_context` | ✅ (for context comparison) |

### 1.2 VCET-side (does NOT exist — first blocker)

| Field | Intended source | Status |
|---|---|---|
| `vcet_state` at entry | `main.vcet_bars` | ❌ **table doesn't exist on dev** (per audit) |
| `reason_code` at entry | `main.vcet_bars` | ❌ same |
| `vcet_score` | `main.vcet_bars` | ❌ same |
| `vcet_grade` (A/B/C/—) | `main.vcet_bars` | ❌ same |
| `vwap_band_width_pct` | `main.vcet_bars` | ❌ same |
| `vwap_slope` | `main.vcet_bars` | ❌ same |
| `price_distance_from_vwap_sd` | `main.vcet_bars` | ❌ same |
| `range_compression_score`, `expansion_score` | engine output | ⚠ **need to verify which are in `REQUIRED_OUTPUT_COLUMNS`** — v1.0 outputs 27 cols, exact names need check. Reason codes carry the underlying signals indirectly. |

### 1.3 Underlying bars (partially blocked)

| Source | Availability | Notes |
|---|---|---|
| `kti price query five_min` (live) | ⚠ **retention-limited** | June 2 probe found bars present May 19+, empty April 21. Today (June 4) the cutoff has rolled forward — April + early-May trades likely have NO bars. |
| `smb.market_bars` (dev DB) | ❌ stale through 2025-10-21 | Not usable for 2026 trades |
| `public.market_data_1min` (dev DB) | ❌ empty | Schema present but 0 rows |
| External (Polygon, Tiingo, IBKR) | ❌ not connected | Memory mentions road map but not today |

**Empirical bar availability estimate** (based on June 2 probe rolled forward 2 days):

| Trade era | n trades | Bar availability estimate |
|---|---:|---|
| April 2026 (1st half) | ~70 | **none** — outside kti window |
| April 2026 (2nd half) | ~50 | **none** |
| May 2026 (1st half) | ~50 | **few/none** — at edge of kti window |
| May 18-31 | ~50 | **most** — within kti window |
| June 1-3 | ~25 | **all** — recent |
| **Usable subset** | **~75-90 trades** | concentrated in last 16 days |

That's the pilot-study sample size. The full 310-row validation would need a historical bar source we don't have. See §3.

---

## 2. Which trades to test first

Prioritized by hypothesis impact + sample size + bar availability.

### 2.1 Phase 4a — pilot (executable on current data)

| Priority | Slug | Total n | Est. usable n (May 18+) | Hypothesis tested |
|---|---|---:|---:|---|
| 1 | `vwap-pull-in` (plain) | 127 | ~35-45 | H2 (leakers in indecision states) |
| 2 | `vwap-pull-in -> trend` | 15 | ~5-7 | H1 (converters in expansion) — **sample too small for confidence** |
| 3 | `chase-extended` | 56 | ~18-22 | H4 (extended-chase context) |
| 4 | `unclassified` | 99 | ~30-35 | secondary — what is the off-book bucket actually showing? |
| 5 | `backside-long` | 13 | ~4-5 | **defer** — n too small |

**Honest sample-size verdict:**
- Plain `vwap-pull-in`: usable. Splits 35-45 trades into winners/losers gives ~15/20 buckets per state — enough to spot strong patterns, not enough for tight statistical claims.
- `vwap-pull-in -> trend`: too small. 5-7 usable trades won't carry confidence. Surface the pattern as suggestive only.
- `chase-extended`: usable for the sample-review question James already flagged.
- `backside-long`: defer until more trades accumulate or VCET backfill lands.

### 2.2 Phase 4b — broader (requires bar-source decision)

To test the full 310-trade sample, we'd need bars for the April–early-May date range. Options:

| Option | Cost | Pros / Cons |
|---|---|---|
| (a) Wait for kti retention growth (if it grows) | $0 | unknown timeline; may never roll backward |
| (b) Sign up + integrate Polygon or Tiingo | $99-$299/mo | full history, but new service to wire |
| (c) Backfill `main.vcet_bars` from whatever bar source we get | engineering | enables the full validation + future joins |
| (d) Accept the pilot-only scope; lock the Quality Gate at RVOL+gap | $0 | proven actionable, no new infra |

This is a James decision, not a Claude decision. The plan executes (a) by default (pilot on whatever's there).

---

## 3. Join logic

### 3.1 Snap rule (same as the original brief, with timezone handling)

For each trade row in `main.trade_context`:

```
1. Convert entry_time (ET naive) → UTC tz-aware:
     entry_utc = entry_time.localize("US/Eastern").astimezone("UTC")

2. Find the VCET bar where:
     bar_timestamp_utc <= entry_utc
   AND
     bar_timestamp_utc > entry_utc - INTERVAL '5 minutes'

3. If no such bar exists:
     mark trade as "NO_BAR" — exclude from VCET-side analyses, retain for sample-count denominator
   Else:
     extract vcet_state, reason_code, vcet_score, vcet_grade, vwap_band_width_pct,
             vwap_slope, price_distance_from_vwap_sd from that bar
```

This is the safety rule from the original VCET brief: never use a bar more than 5 minutes old (prevents stale-bar contamination on halted/illiquid symbols), never use a future bar (prevents look-ahead leakage).

### 3.2 State-before / state-after

For each entry:
- **State -1 bar (5 min before entry)**: `bar_timestamp_utc <= (entry_utc - INTERVAL '5 minutes') AND bar_timestamp_utc > (entry_utc - INTERVAL '10 minutes')`
- **State +5 bars (25 min after entry, ~typical hold time)**: `bar_timestamp_utc > (entry_utc + INTERVAL '25 minutes') AND bar_timestamp_utc <= (entry_utc + INTERVAL '30 minutes')`

These give the state trajectory around the entry — does the state CHANGE between just-before and at-entry (often the trigger of the trade)?

### 3.3 Source of VCET bars (3.1's `bar_timestamp_utc`)

For Phase 4a pilot:
- Fetch bars from kti at runtime per unique (symbol, date) in the sample
- Compute VCET states in memory using `vcet_engine.run_vcet(bars, symbol, '5m')` with **slope_trend_thresh = 2.0** (the calibrated value from earlier sweep)
- Snap to entry per §3.1
- Do NOT persist to `main.vcet_bars` (table doesn't exist; we'd need DDL + James approval first)

This mirrors the approach `validate_best_trades.py` already took on June 1 — but with:
- All 310 trades (or whatever subset has bars) instead of the original 60-trade sample
- Canonical-slug keys (now possible post-Phase-2) instead of display-name keys
- Three state checkpoints (entry, -5min, +25min) instead of just entry

---

## 4. Report outputs

The validation produces these tables. All numbers are illustrative; real values land at execution.

### 4.1 Strategy × VCET state matrix

```
canonical_slug          | VCET state at entry              | n  | total_pnl | avg   | win%
------------------------+----------------------------------+----+-----------+-------+------
vwap-pull-in -> trend   | EARLY_EXPANSION                  | 3  | +$67      | +$22  | 100%
vwap-pull-in -> trend   | EXPANSION_CONFIRMED              | 2  | +$45      | +$22  | 100%
vwap-pull-in -> trend   | TREND_UP                         | 1  | +$12      | +$12  | 100%
vwap-pull-in -> trend   | ROTATION                         | 0  | —         | —     | —
vwap-pull-in            | ROTATION                         | 18 | -$140     | -$7.8 | 28%
vwap-pull-in            | FAILED_EXPANSION_TENTATIVE       | 8  | -$48      | -$6   | 25%
vwap-pull-in            | EARLY_EXPANSION                  | 6  | +$22      | +$3.7 | 67%
vwap-pull-in            | EXPANSION_CONFIRMED              | 3  | +$31      | +$10  | 100%
...
```

**Read:** plain Pull-In aggregated by state shows where the leak lives. If H2 holds, ROTATION + FAILED_EXPANSION are concentrated bad.

### 4.2 Pull-In split by VCET state (the key comparison)

```
VCET state                | PI -> Trend n | PI -> Trend win% | Plain PI n | Plain PI win% | Δ win%
--------------------------+---------------+------------------+------------+---------------+--------
ROTATION                  | 0             | —                | 18         | 28%           | n/a
COMPRESSION               | 0             | —                | 4          | 50%           | n/a
EARLY_EXPANSION           | 3             | 100%             | 6          | 67%           | +33 pp
EXPANSION_CONFIRMED       | 2             | 100%             | 3          | 100%          | 0
TREND_UP                  | 1             | 100%             | 2          | 50%           | +50 pp
FAILED_EXPANSION_TENT     | 0             | —                | 8          | 25%           | n/a
MEAN_REVERSION            | 0             | —                | 1          | 0%            | n/a
NO_BAR                    | 9             | 89%              | 84         | 36%           | (excluded)
```

**Read:** if PI→Trend lives in expansion states and plain PI lives in ROTATION + FAILED_EXPANSION, the hypothesis is supported. The "NO_BAR" row is the historical sample we can't classify — it's the case for the bar-source decision in §2.2.

### 4.3 Winners vs losers by VCET state (across all slugs)

```
VCET state                | n winners | n losers | winner avg pnl | loser avg pnl
--------------------------+-----------+----------+----------------+---------------
EARLY_EXPANSION           | 18        | 4        | +$15.2         | -$8.0
EXPANSION_CONFIRMED       | 11        | 2        | +$22.1         | -$5.5
TREND_UP                  | 5         | 1        | +$18.4         | -$3.2
ROTATION                  | 14        | 28       | +$3.8          | -$6.1
FAILED_EXPANSION_TENT     | 3         | 11       | +$4.0          | -$8.5
COMPRESSION               | 4         | 3        | +$5.5          | -$4.0
MEAN_REVERSION            | 2         | 5        | +$3.0          | -$10.5
```

**Read:** independent of strategy slug, do certain VCET states predict win-rate? This is the macro test of whether VCET is useful at all as a state-based filter.

### 4.4 State trajectory (before → at → after entry)

```
Trajectory pattern                                    | n  | total_pnl | win%
------------------------------------------------------+----+-----------+-----
ROTATION → EARLY_EXPANSION → EXPANSION_CONFIRMED       | 8  | +$165     | 88%
ROTATION → ROTATION → ROTATION                          | 24 | -$180     | 25%
COMPRESSION → EARLY_EXPANSION → TREND_UP                | 4  | +$92      | 100%
EARLY_EXPANSION → FAILED_EXPANSION_TENT → ROTATION     | 6  | -$72      | 17%
```

**Read:** trajectory tells the story better than state-at-entry alone. PI→Trend setups should show a transition INTO expansion; plain PI losses might show transitions OUT of expansion back to rotation.

### 4.5 Band geometry comparison (numeric)

```
canonical_slug × outcome    | n  | avg vwap_band_width_pct | avg vwap_slope | avg dist_from_vwap_sd
----------------------------+----+-------------------------+----------------+----------------------
PI -> Trend / W             | 14 | 1.42                    | +3.2           | +0.6
plain PI / W                | 45 | 1.18                    | +1.8           | +0.4
plain PI / L                | 80 | 0.95                    | +0.4           | -0.1
chase-extended / W          | 32 | 1.85                    | +5.1           | +1.8
chase-extended / L          | 24 | 1.65                    | +4.2           | +2.4
```

**Read:** if band geometry differs systematically between winners and losers within the same slug, that's a numeric rule beyond states (RVOL+gap might be a proxy for this).

### 4.6 Top rule candidates (synthesized at execution)

The execution produces a final §"Rule candidates" similar to the Pull-In diagnostic — concrete RVOL-style rules but using VCET state/score. E.g.:

> **Rule V1.** Skip plain Pull-In if VCET state at entry is in {ROTATION, FAILED_EXPANSION_*} AND vcet_score < 50.
> **Rule V2.** Hold Pull-In for trend conversion if entry state is EARLY_EXPANSION AND band_width > 1.2 AND slope > +2.0.
> **Rule V3.** For chase-extended, demand state in {EXPANSION_CONFIRMED, TREND_UP} — pass otherwise.

Numbers are illustrative; real ones land at execution.

---

## 5. Visual output

### 5.1 New report

**File:** `Z:\public_html\system\VCET_AT_ENTRY_VALIDATION_REPORT.html`
**URL:** `https://trsportsacademy.com/system/VCET_AT_ENTRY_VALIDATION_REPORT.html`

Same structure / style as `CANONICAL_STRATEGY_PERFORMANCE_REPORT.html`:
- Self-contained HTML, inline CSS, dark theme
- Static snapshot (no live DB binding in the browser)
- Mobile-respectable layout

### 5.2 Sections

1. **Executive summary** — 5 bullets: which hypotheses held, which didn't, the rule candidates that emerged, sample-size caveats
2. **Sample availability** — what % of trades had usable bars, broken down by week, with the NO_BAR row explicit
3. **Strategy × VCET state matrix** (the §4.1 table)
4. **Pull-In split by VCET state** (the §4.2 table, with delta column color-coded)
5. **Winners vs losers across all slugs** (the §4.3 table)
6. **State trajectory patterns** (the §4.4 table, with top 8 trajectories by n)
7. **Band geometry by outcome** (the §4.5 table)
8. **Rule candidates V1–Vn** with estimated P&L impact
9. **Limitations** — small samples, kti retention, no v1.1/CBO, single threshold value tested
10. **Links** — back to Pull-In diagnostic, Quality Gate, and the per-slug performance report

### 5.3 Color-coded cards

For each canonical slug, a card at the top showing:
- Slug name + badge (edge/leak/sentinel)
- All-time n + P&L (carry-forward from existing report)
- **State distribution mini-bar**: horizontal stacked bar showing what % of trades fall into each VCET state (color-coded: green for trend states, yellow for early expansion, gray for rotation, red for failed expansion)
- One-line summary: "76% of wins in expansion states; 64% of losses in rotation"

### 5.4 Linking from CANONICAL_STRATEGY_PERFORMANCE_REPORT.html

Per James's "if safe" — yes, safe. The existing report's §7 already has a list of workflow artifacts. Adding one more link is non-destructive: I'd add a new `<a href="./VCET_AT_ENTRY_VALIDATION_REPORT.html">` block to the §7 `.links` div. No navigation changes, no structural changes. Approved as part of plan execution.

---

## 6. Risks & blockers (the honest part)

### 6.1 Blocker — VCET tables don't exist on dev

`main.vcet_bars`, `main.vcet_state_transitions`, `main.vcet_parity_audit` were all expected by the original brief but **none exist** on `trsports_tradingjournal_dev` (confirmed in earlier audit).

**Impact on Phase 4:**
- The plan's §3 join logic CANNOT join against `main.vcet_bars` — it doesn't exist.
- **Workaround:** compute VCET states in-memory at validation time. Fetch bars from kti per (symbol, date), run `vcet_engine.run_vcet()` in a Python script (like `validate_best_trades.py` already does), snap states per §3.1, never persist to DB.
- This is read-only; no DB modifications.
- **Caveat:** we'd recompute VCET every time we re-run the validation. Slow but acceptable for a one-time validation report.

### 6.2 Blocker — kti retention

Bars are only available for ~14–30 days. For the 310 trade rows spanning April 10 → June 3, only the last 16-20 days (~75-90 trades) have bars. Older trades will fall into the NO_BAR bucket and be excluded from VCET-side analyses.

**Impact:** the validation is a pilot on a recent subset, not a full historical study. Statistical confidence is correspondingly lower.

**Mitigation:** report sample sizes prominently. Don't claim what the small samples don't support.

### 6.3 Risk — v1.0 vs v1.1/CBO confusion

We have v1.0 only. v1.1 (with CBO) never landed. The original brief framed Phase 4 in terms of v1.1 states; we'd be running v1.0 (no CBO).

**Impact:**
- The 9 v1.0 states are still emitted correctly.
- The CBO confirmation layer is absent — Range Break / Hitchhiker classification would be 90% answer, not 100%.
- For the Pull-In split test, CBO isn't on the critical path — the EARLY_EXPANSION → EXPANSION_CONFIRMED → TREND_UP transitions don't require CBO.

**Mitigation:** explicitly state the report is v1.0-based. Note that v1.1 would refine `chase-extended` classification specifically.

### 6.4 Risk — timestamp alignment

`main.trade_context.entry_time` is naive TIMESTAMP. The code assumes ET local. VCET bars from kti are UTC.

**Mitigation:** §3.1 converts explicitly (`.localize("US/Eastern").astimezone("UTC")`). Test this conversion on 3 known trades before running the full set.

### 6.5 Risk — small samples → low statistical confidence

`vwap-pull-in -> trend` has only 15 total trades; the usable VCET subset is likely 5-7. Any pattern there will be suggestive, not decisive.

**Mitigation:** report 95% CIs on key percentages. State the suggestive vs decisive distinction explicitly. Treat the 5-7-trade PI→Trend result as "directional evidence" not "validated rule."

### 6.6 Risk — slope_trend_thresh tuning still in play

Earlier we calibrated slope_trend_thresh=2.0 as the right value for NVDA-class names. The validation will use 2.0. But individual trades on different liquidity tiers might warrant different thresholds. v1.1's ATR-relative slope would be the proper fix.

**Mitigation:** report results at 2.0; note that re-running with per-symbol-tier thresholds is a follow-on once Phase 4 establishes the baseline.

---

## 7. Execution sequence (if approved)

In order. Each step has a clean stop point.

| Step | What | Time | Output |
|---|---|---|---|
| 0 | James approves this plan + chooses bar-source path (default: kti-only pilot) | — | go / adjust scope |
| 1 | Write `python/validate_vcet_at_entry.py` (a Phase-4-specific script; not modifying existing scripts) | 30-60 min | new file at `Z:\setup_daily_reporting\reports\` |
| 2 | Pilot run on 5-symbol subset (NVDA, SPY, QQQ, AAPL, MSFT) over last 14 days to verify the join logic works end-to-end | 5 min | console output, sanity-check NO_BAR rate |
| 3 | Full pilot run on all 310 trades (will produce NO_BAR for older trades) | 5-10 min | CSV / JSON of per-trade VCET tags |
| 4 | Build the static HTML report | 10 min | `VCET_AT_ENTRY_VALIDATION_REPORT.html` |
| 5 | Add the link to `CANONICAL_STRATEGY_PERFORMANCE_REPORT.html` §7 | 1 min | one-line edit |
| 6 | Report results to James | — | summary message + URL |

**Stop after each step if you want to inspect output.** No automatic chaining.

---

## 8. Acceptance criteria for Phase 4 outputs to be useful

| Outcome | Decision |
|---|---|
| Usable sample is < 30 trades on each side of the Pull-In split | Report as suggestive only; recommend Phase 4b (acquire historical bars) before locking rule candidates |
| Usable sample 30-80 trades and patterns are clear | Lock 1-3 rule candidates; add them to the Quality Gate as v2 |
| Usable sample 80+ and patterns are statistically clean | Lock rules; recommend Phase 5 (operationalize rules into `gen_scorecard.py` classifier or pre-entry filter) |
| Patterns are noisy / VCET states don't discriminate | Accept that VCET v1.0 isn't the right layer; Quality Gate (RVOL+gap) remains the operational filter; reconsider Phase 4 after v1.1/CBO lands |

---

## 9. What this plan does NOT propose

- **NO Phase 4 execution yet.** This is plan-only.
- **NO VCET code edits.** `vcet_engine.py` stays untouched.
- **NO `main.vcet_bars` table creation.** That would require DDL approval; out of scope here.
- **NO modification of `main.trade_context`** schema or data.
- **NO Scorecards refresh.** Phase 3b remains on hold.
- **NO taxonomy changes.** Phase 2's canonical slugs are stable.
- **NO production touches.** Dev/staging only.
- **NO v1.1 / CBO work.** Still on hold pending the real v1.1 zip.
- **NO new Python deps.** Reuses `psycopg2`, `pandas`, `subprocess` already in use.

---

## 10. The actual question Phase 4 answers (one line)

**Does the regime VCET claims to detect at entry empirically predict which Pull-Ins convert to trend versus which leak?**

If yes → VCET state becomes the strongest filter in your pre-trade gate, displacing RVOL+gap as primary.
If no → VCET v1.0 is the wrong layer for this question; the Quality Gate (RVOL+gap) remains primary and you re-evaluate when v1.1/CBO lands.

Either answer is a tangible decision. The plan is ready when you say go.
