Overview

Stack: Python 3.12 · FastAPI · React 19 · PostgreSQL · PyOD · Prophet · XGBoost · FinBERT · MLflow

What It Does

Sentinel ingests financial data from multiple public sources, engineers features, trains and serves ML models (anomaly detection, forecasting, sentiment), and exposes everything through a REST API and a Bloomberg-style React dashboard.

The backend is a single FastAPI process backed by PostgreSQL (SQLAlchemy 2.0). The frontend is a decoupled React SPA served separately.

What’s Real

Every pipeline, model, endpoint, and dashboard page listed in these docs runs today:

Design Choices

• Decoupled frontend. React SPA on its own domain, API on another. CORS fully open for development.
• hermes-plt integration. Technical and fundamental analysis pull from the Hermes library for deep market data.
• CLI-driven orchestration. Ingestion and training are stl commands; no scheduler yet.
• Environment-driven configuration. Everything lives in config/settings.py, read from .env.

See Architecture for the full layout.

Architecture

                 ┌──────────────────────────────┐
                 │        Backend               │
                 │   (uvicorn → FastAPI :7860)  │
                 │                              │
  Browser ──────►│  /api/v1/*    FastAPI        │
                 │               routers        │
                 │                              │
                 │  /api/docs   Swagger UI      │
                 └──────────────┬───────────────┘
                                │
                 ┌──────────────▼───────────────┐
                 │   PostgreSQL (SQLAlchemy)    │
                 │  Supabase / local / managed  │
                 └──────────────────────────────┘

  Frontend: React 19 + Vite 8 + TypeScript + Plotly.js
  Deployed separately (Render free-tier)

  Data sources: yfinance · CCXT · FRED · newsdata.io · hermes-plt
  ML: PyOD · Prophet · XGBoost · FinBERT · scipy   → MLflow registry
  MLOps: drift detection (PSI + KS) → email/Slack alerts

Project Layout

Runtime Flow

1. Ingest — stl ingest pulls OHLCV, crypto, macro, news, and analysis data into PostgreSQL.
2. Train — stl ml-train fits anomaly/forecast/sentiment models and registers them in MLflow.
3. Serve — stl api-start runs uvicorn on port 7860; frontend runs separately via Vite or is deployed to Render.

CLI (stl)

Getting Started

Requirements

• Python 3.12 (pinned via .python-version)
• uv for dependency management
• A reachable PostgreSQL (local, Docker, or Supabase pooler URL)
• Node.js 18+ (for the frontend)

Install Backend

git clone <repo-url> Sentinel
cd Sentinel
uv sync

Configure

cp .env.example .env

Initialize Schema

Option A — Supabase: paste database/schema.sql then database/seed_data.sql into the SQL Editor.

Option B — any Postgres:

psql "$DB_URL" -f database/schema.sql && psql "$DB_URL" -f database/seed_data.sql

Run

stl all    # ingest → train → start API

Or individually:

stl ingest
stl ml-train
stl api-start

Install Frontend

cd web
npm install
npm run dev    # http://localhost:5173 (proxies /api/* to :7860)

Verify

Data Ingestion

Files: ingestion/

Fetchers

Analysis Data

analysis_data.py uses the hermes-plt library for deep market data:

• Technical: 20 crypto symbols via hr.ta_feature.get_technical() — OHLCV, returns, MAs, EMAs, volatility, order book, funding rate, open interest, composite scores
• Fundamental: 18 US equities via hr.fa_features.get_fundamentels() — income statement, balance sheet, cash flow, valuation ratios, macro indicators, earnings estimates

Feature Engineering

feature_engineer.py runs after stock price ingestion. For every ticker in the DB:

• Log returns
• Lag features (1d, 5d, 21d, 63d)
• Rolling statistics (21-day window): mean, std, skew
• RSI (14-period) via EWM
• Bollinger %B (20-day, 2 std)
• Volume ratio (20-day rolling average)

Pipeline Runner

ingestion_pipeline.py runs every stage in sequence with pass/fail reporting:

stl ingest

Or run individual stages:

python -m ingestion.price_fetcher
python -m ingestion.crypto_fetcher
python -m ingestion.macro_fetcher
python -m ingestion.news_fetcher
python -m ingestion.analysis_data
python -m ingestion.feature_engineer

Each pipeline exits non-zero on failure.

Machine Learning

Files: ml/

Anomaly Detection

anomaly_detector.py — Isolation Forest + Local Outlier Factor (PyOD ensemble).

• Features: log_return, volume_ratio, rolling_std_21, rsi_14, bb_pct_b
• Contamination: 5% expected anomalies
• Process: StandardScaler → IForest (100 estimators) + LOF (20 neighbors) → MinMax-normalize and average → severity labels (≥0.8 critical, ≥0.6 high, ≥0.4 medium, below 0.4 low)
• Default tickers: AAPL, MSFT, NVDA, TSLA, GOOGL

Forecasting

forecaster.py — Prophet + XGBoost blended 50/50.

• Horizons: 30 days and 90 days
• XGBoost features: lag_1d, lag_5d, lag_21d, lag_63d, rolling_mean_21, rolling_std_21, rsi_14, volume_ratio, bb_pct_b
• Cross-validation: Walk-forward CV (5 splits), MAE + RMSE
• Blending: Simple average of Prophet yhat and XGBoost predictions

Sentiment Analysis

sentiment_engine.py — aggregates historical sentiment scores.

• Daily averages with 7-day and 30-day rolling means
• Signal: 7d > 30d = BULLISH, 7d below 30d = BEARISH, else NEUTRAL
• Note: FinBERT scoring happens in ingestion, not here

Portfolio Optimization

portfolio_optimizer.py — four allocation methods:

Tickers: AAPL, MSFT, NVDA, TSLA, GOOGL. Risk-free rate: 5%, 252 trading days.

Feature Store

feature_store.py — in-memory dict cache of feature DataFrames per ticker, with DB fallback.

Model Registry

model_registry.py — MLflow integration: save, load, promote models with metrics/params/artifacts tracked per run.

Training Pipeline

stl ml-train

Two modes:

• run_pipeline() — basic: warmup → anomalies → forecasting → sentiment → portfolio
• run_pipeline_with_metrics() — enhanced: collects CV metrics, logs to MLflow (enabled via WITH_METRICS=true)

MLOps & Drift

File: mlops/drift_detector.py (411 lines)

What It Does

Runs statistical tests per feature against a rolling baseline, catching distribution shifts before they degrade model performance.

Windows

Monitored Features (11)

Statistical Tests

Alerting

Drift triggers email + Slack alerts with detailed reports:

• Email: HTML-formatted, includes drifted features, PSI values, baseline vs production means
• Slack: Block Kit formatting via webhook

CLI

python mlops/drift_detector.py --tickers AAPL,MSFT --send-alerts

• Exit code 1 if drift detected (CI-friendly), 0 if stable

Roadmap

• Persist drift runs and expose via API
• Visualize drift in the dashboard
• Add Prometheus/Grafana observability

API Reference

Base: /api/v1 — interactive docs at /api/docs, health at /api/health

Health

Prices (/api/v1/prices)

Anomalies (/api/v1/anomalies)

Forecasts (/api/v1/forecasts)

Sentiment (/api/v1/sentiment)

Portfolio (/api/v1/portfolio)

Analysis (/api/v1/analysis)

Smoke Checks

curl http://localhost:7860/api/health
curl http://localhost:7860/api/v1/prices/AAPL/history?limit=5
curl http://localhost:7860/api/v1/analysis/technical/BTCUSDT
curl http://localhost:7860/api/v1/analysis/fundamental/AAPL

Dashboard

Files: web/src/pages/

The dashboard is a React 19 SPA with Plotly.js charts, lazy-loaded via React.lazy(). Uses a custom dark Bloomberg-style design system with IBM Plex Mono/Sans fonts.

Pages

Sidebar

• Health status indicator (LIVE/DEGRADED/OFFLINE) polled every 30s
• “Wake Up” button with 60s countdown (Render free-tier cold starts)
• Responsive: icon-rail at 980px, bottom tabs at 640px

Theme

Custom dark palette — IBM Plex Mono/Sans typography, terminal-style accents, Plotly chart theming via web/src/theme.ts.

Deployment

Backend

The backend runs as a single uvicorn process on Render free-tier (https://sentinel-ggi8.onrender.com).

• No Dockerfile — Render auto-detects Python and runs uvicorn api.main:app
• Health check on /api/health
• JSON logging with rotating file handler (fails silently on read-only containers)

Frontend

The React SPA is deployed separately on Render as a static site.

• Built with npm run build → dist/
• In development: Vite dev server at :5173 with proxy to backend at :7860
• In production: hits VITE_API_URL directly

Database

PostgreSQL is the only external dependency. Any of these work:

• Local Postgres
• Docker container
• Supabase hosted Postgres (pooler URL, sslmode=require)

All state lives in Postgres — ephemeral filesystem storage is fine for free-tier deployments.

Configuration

All config is environment-driven (config/settings.py, .env):

What’s Not Done Yet

• No Dockerfile (planned for Phase 4)
• No CI/CD (GitHub Actions workflow exists but is empty)
• No unit or integration tests
• No auth or rate limiting (auth code is commented out)
• No scheduled retraining (manual CLI only)