System Architecture

OTM-Agent wraps a three-stage operational pipeline with an LLM-orchestrated agent layer comprising a planner, executor, four-tier memory, Bayesian persona vectors, and multi-objective reward.

High-Level Data Flow

Base Pipeline: Three Stages

Stage 1: SAR Processing & State Estimation

Raw Sentinel-1 SAR observations are processed through confidence scoring (quality weights based on acquisition geometry, weather, temporal recency), z-score normalization, and Kalman filtering to produce daily hub-state estimates — a tightness index (0 = empty, 1 = full) with associated uncertainty.

Stage 2: Feature Engineering & ML Models

Market data (WTI prices, calendar spreads, COT positions, EIA releases) is joined with Stage 1 hub-state output to produce linkage features. Three LightGBM classifiers consume these features:

Stage 3: Signal Generation & Governance

A signal generator combines model outputs into daily composite trading signals (long/short/flat) with confidence. A paper trading module executes simulated trades, and a governance scorecard tracks prediction accuracy.

Agent Layer

Turn Flow

A single agent turn proceeds through 9 steps:

1. Context assembly — populate WorkingMemory with query + context
2. Persona retrieval — load 64-d persona vector for current user
3. State signature — MD5 hash of regime × volatility × persona mode
4. Memory retrieval — 3 episodic + 5 semantic + 2 procedural sequences
5. Plan generation — LLM produces structured JSON plan (goal, rationale, skill steps)
6. Plan validation — skill registry checks preconditions, costs, DAG constraints
7. Execution — dispatch skills, track effects, replan on failure (up to 3 replans)
8. Response generation — synthesize natural-language response from skill outputs
9. Memory persistence — record episodic entry, extract semantic facts, update persona

Skill Library — 20 Skills, 7 Categories

Skill Contract Example

{
  "name": "predict_regime",
  "category": "ML Models",
  "preconditions": ["linkage_features_available"],
  "effects": ["regime_prediction_available"],
  "failure_modes": ["insufficient_data", "model_load_error"],
  "avg_latency_ms": 850,
  "avg_cost_usd": 0.001,
  "retention_probes": [
    "What regime did the model predict?",
    "What was the classification confidence?",
    "Which feature contributed most?"
  ]
}

Four-Tier Memory System

State signatures are 8-character MD5 hashes of (regime × volatility × persona mode), enabling transfer across sessions with comparable market conditions.

Persona Module — 64-Dimensional Bayesian Vectors

Each analyst's preferences are represented as a 64-dimensional vector maintained as a Bayesian posterior (mean + 64×64 covariance). Two anchor directions define the primary axis:

After each interaction, the persona vector is updated via Kalman-style Bayesian inference. Over multiple turns, the covariance shrinks and behavior becomes increasingly specialized for the individual analyst.

Infrastructure