Stage A.1 - Liberator Timestamp Normalization

Run date 2026-06-07 · Stage A.1 §2 deliverable · Based on 5 cached Liberator samples (NVDA 6/2; AAOI 4/10; CAR/WULF/CRWV 4/13) · Anchor: Pull-In Research Hub

Verdict: Liberator timestamps are end-of-bar with millisecond precision; normalize by subtracting 59.999 seconds

Liberator emits timestamps in the form YYYY-MM-DD HH:MM:59.999 (end of the 60-second bar). kti emits timestamps as YYYY-MM-DDTHH:MM:00 (start of bar). Both represent the same 60-second window. The Liberator-to-engine normalization rule is: parse as US/Eastern naive, subtract 59.999 seconds, localize to ET, convert to UTC. Verified against the cached Liberator + kti pairs.

1. Evidence: Liberator timestamps are end-of-bar

1.1 Raw Liberator timestamps (first 5 bars of NVDA 2026-06-02)

2026-06-02 04:00:59.999     # bar 1 - covers 04:00:00 to 04:01:00 ET
2026-06-02 04:01:59.999     # bar 2 - covers 04:01:00 to 04:02:00 ET
2026-06-02 04:02:59.999
2026-06-02 04:03:59.999
2026-06-02 04:04:59.999

1.2 Raw kti timestamps (same NVDA 2026-06-02)

2026-06-02T09:30:00         # bar - covers 09:30:00 to 09:31:00 ET
2026-06-02T09:31:00
2026-06-02T09:32:00
2026-06-02T09:33:00
2026-06-02T09:34:00

1.3 Conclusion

SourceFormatAnchorPrecision
LiberatorYYYY-MM-DD HH:MM:59.999End of barMillisecond
ktiYYYY-MM-DDTHH:MM:00Start of barSecond

Both represent the same 60-second window. To align Liberator with kti and the v1.1 engine's compute_features() expectations (which treat bar_timestamp as the bar's start), Liberator needs to be normalized.

2. Normalization rule

Three equivalent formulations:

2.1 Pandas (the form used in stage_a1_parity.py)

# Liberator emits: "YYYY-MM-DD HH:MM:59.999" naive ET
parsed = pd.to_datetime(lib_df["timestamp"])

# Subtract 59.999 seconds to get the START of the bar
bar_start = parsed - pd.Timedelta(seconds=59, milliseconds=999)

# Localize to US/Eastern, then convert to UTC (engine expectation)
lib_df["timestamp"] = bar_start.dt.tz_localize("US/Eastern").dt.tz_convert("UTC")

2.2 Equivalent: floor to minute

# Floor each timestamp to the nearest minute below; same result
bar_start = parsed.dt.floor("min")
lib_df["timestamp"] = bar_start.dt.tz_localize("US/Eastern").dt.tz_convert("UTC")

Note: floor("min") on HH:MM:59.999 returns HH:MM:00 - exactly the kti start-of-bar form. This is the cleaner one-liner.

2.3 Equivalent: subtract 60 seconds and add 1ms (paranoid form)

# Slightly more paranoid: subtract 60s, add 1ms to land exactly on minute boundary
bar_start = parsed - pd.Timedelta(seconds=60) + pd.Timedelta(milliseconds=1)

Use floor("min") in production. The other forms are listed for clarity.

3. Before / after example

3.1 Before normalization (raw Liberator + kti, naive)

SourceTimestamp stringParsed
kti2026-06-02T09:30:002026-06-02 09:30:00 (naive ET)
Liberator2026-06-02 09:30:59.9992026-06-02 09:30:59.999 (naive ET)

Direct merge would fail - the strings don't match.

3.2 After normalization

SourceOriginalNormalized (UTC)
kti2026-06-02T09:30:00 (ET)2026-06-02 13:30:00+00:00
Liberator (after subtract 59.999s + tz)2026-06-02 09:30:59.999 (ET)2026-06-02 13:30:00+00:00

Match exact. The pandas merge on timestamp column will join on the same key.

4. Verification on cached pairs

Applied the normalization rule to 5 cached Liberator pairs. Compared against kti where overlap existed (only NVDA 6/2):

PairLib raw barsLib normalized RTH barskti RTH barsTimestamp alignment
NVDA 2026-06-02960390390All 390 align exactly
AAOI 2026-04-10957390(kti has no bars; older than window)No kti to compare
CAR 2026-04-13760390(no kti)—
WULF 2026-04-13838390(no kti)—
CRWV 2026-04-13960390(no kti)—

All 5 produce exactly 390 RTH bars after normalization and RTH filtering. The single overlap pair (NVDA 6/2) aligns 100% with kti. The normalization rule is sound.

5. Pitfalls

PitfallSymptomFix
Forgetting timezone localizationNaive ET parsed as naive UTC (4-hour shift wrong direction)Always .tz_localize("US/Eastern") after parsing
Not subtracting 59.999sLiberator bar that covers 09:30-09:31 ET gets timestamp 09:31:00 UTC; off-by-one bar; merges missUse floor("min") after parsing
Using round("min") instead of floorHH:MM:59.999 rounds UP to HH:(MM+1):00Use floor not round; or subtract 1 second first
Mixing Liberator's _seq ordering with timestamp ordering_seq is a global counter, not a per-symbol time indexAlways sort by parsed timestamp, never by _seq
Liberator bars sometimes have 0 vwap at premarket boundaryVWAP = 0 in the first premarket bar of the dayFilter or set to NaN; do not divide

6. Recommended Stage B loader (the spec)

def liberator_to_df(bars: list) -> pd.DataFrame:
    """Normalize Liberator bars to start-of-bar UTC, kti-compatible schema."""
    df = pd.DataFrame(bars)
    parsed = pd.to_datetime(df["timestamp"])
    bar_start = parsed.dt.floor("min")
    df["timestamp"] = (
        bar_start
        .dt.tz_localize("US/Eastern")
        .dt.tz_convert("UTC")
    )
    df = df[["timestamp", "open", "high", "low", "close", "volume", "vwap"]].copy()
    for c in ("open", "high", "low", "close", "vwap"):
        df[c] = df[c].astype(float)
    df["volume"] = df["volume"].astype(float)
    return df.sort_values("timestamp").reset_index(drop=True)

The vwap column is Liberator's native bar-level VWAP and is included for the §3 Liberator-VWAP-cross-check (deferred until entitlement is restored - see parity report).

Links