Foresea Forecasting

Foresea: Autonomous Prediction Market Intelligence & LLM Rationale Benchmark

Foresea (foresea.ink) is an autonomous prediction market intelligence platform and empirical research framework. It combines real-time probability forecasting across Polymarket and Kalshi, statistical edge discovery, multi-model shadow trading tournaments, an autonomous 19-tool ReAct execution agent, and a public Model Context Protocol (MCP) server.

The repository also serves as the artifact for research studying how explicit rationale instructions, evidence injection, and reasoning structures affect LLM forecasting behavior and calibration on Metaculus-style questions.

---

📑 Table of Contents

• Live Platform & Web Views
• Edge Board & Autonomous Shadow Trading Desk
• 5-Minute Crypto Micro-Markets
• Production Deployment & Infrastructure
• Using the API & Endpoints
• Agent: Automated Intelligence Layer & ReAct Tools
• Model Context Protocol (MCP) Integration
• Custom Integrations & Ecosystem Tools
• Trading Execution & KMS Guardrails
• Repository Contents & Prompt Variants
• Foresea Autoresearch
• Reproducing Core Outputs & SQL Analytics
• Quality Checks & Development

---

Live Platform & Web Views

Deployed on Google Cloud Run with high-concurrency scaling, streaming responses, and continuous delivery:

https://foresea.ink

Core Web Applications & Routes

• Market Desk (`/`): Real-time landing view with market radar, model-vs-market gap highlights, and interactive walkthroughs.
• Forecasting Chat (`/chat`, `/chat/:id`): Conversational interface with streaming rationale, news evidence citation, and multi-turn market analysis.
• Edge Board (`/edge`, `/edge/markets`): Ranked live Foresea-vs-market pricing discrepancies across Polymarket and Kalshi, backed by calibration and lead-time scores.
• Mark-to-Market View (`/edge/mtm`): Continuous mark-to-market valuation and PnL tracking for resolved and open market predictions.
• Agentic Trading Desk (`/edge/agentic`): Multi-model autonomous trading tournament. Independent $10,000 shadow accounts for each model (Gemma 4 26B, Qwen 3.8 27B, GPT-OSS 120B, GLM 5.3, GLM 5.3 Flash, DeepSeek V4 Flash, MiniMax M3, Llama 3.3 70B) executing real-time paper trades, tracking equity curves, and logging hourly cycle health.
• Trading Terminal (`/trade`): Non-custodial, client-side order preview and execution terminal with Cloud KMS envelope encryption for Polymarket and Kalshi credentials.
• Watchlist (`/watchlist`): Follow specific markets with automated daily digest emails.
• Public Forecasts (`/forecast/:share_id`): Shareable forecast permalinks with rationale cards and provenance.

# Health check

curl https://foresea.ink/health

Single-record prediction

curl -X POST https://foresea.ink/predict \ -H "Content-Type: application/json" \ -d '{ "question": "Will X happen by date Y?", "question_type": "binary", "description": "Context here.", "news_articles": [], "attach_evidence": true, "evidence_top_k": 5, "market_platform": "Polymarket", "market_probability": 0.42, "variant": "variant0_neutral_baseline" }'

When attach_evidence is true and no news_articles are supplied, /predict fetches and ranks current news evidence from GDELT, Google News RSS, and Stooq by default, injects it into the model prompt, and returns the selected evidence_articles with the forecast. Supplying news_articles skips automatic retrieval and uses the caller-provided evidence.

The response includes both the forecast and the evidence used by the model:

{

"question_type": "binary", "predicted_answer": "Yes", "confidence": 0.86, "options": [], "range_forecast": null, "rationale": "Model-generated explanation for the forecast.", "model_rationale": "Model-generated explanation for the forecast.", "variant": "variant0_neutral_baseline", "model_key": "gpt-oss-120b", "evidence_sources": [ { "source": "Reuters", "title": "Article headline", "url": "https://example.com/article", "publish_date": "2026-05-29T00:00:00Z", "relevance_score": 0.82 } ], "evidence_articles": [ { "title": "Article headline", "summary": "Cleaned article summary.", "source": "Reuters", "url": "https://example.com/article", "publish_date": "2026-05-29T00:00:00Z", "relevance_score": 0.82, "search_query": "query used for retrieval" } ], "evidence_error": null, "market_analysis": { "platform": "Polymarket", "market_url": "https://example.com/market", "outcome": "Yes", "market_probability": 0.42, "model_probability": 0.86, "edge": 0.44, "stance": "model_above_market", "summary": "Foresea is 44 percentage points above the market on Yes." } }

Use evidence_sources when a client only needs the source list and links. Use evidence_articles when a client needs the article-level details that were attached to the model prompt. rationale and model_rationale are generated by gpt-oss-120b and explain why the model chose its answer and confidence. When market_probability is supplied, market_analysis is computed deterministically from the model probability and the market-implied probability.

Edge Board & Autonomous Shadow Trading Desk

Foresea continuously evaluates state-of-the-art LLMs against real-world prediction markets, tracking statistical edge, calibration accuracy, and shadow portfolio performance across three dedicated views:

1. Live Markets Disagreement (/edge or /edge/markets)

• Real-Time Edge Ranking: Quantifies the statistical gap between model forecasts and live market pricing across Polymarket and Kalshi: $$\text{Edge} = P_{\text{model}} - P_{\text{market}}$$
• Sizing & Recommendation: Computes Fractional Kelly allocations and categorizes actionable trades (buy_yes, buy_no, hold).
• Calibration & Resolution: Backed by hourly snapshots committed to static/track_record_live.json, calculating Brier scores, Expected Calibration Error (ECE), and lead-time skill once markets resolve.

2. Mark-to-Market Evaluation (/edge/mtm)

• Continuous Portfolio Valuation: Evaluates paper positions at live bid/ask quotes rather than relying on eventual settlement alone.
• Unrealized vs. Realized PnL: Provides an institutional-grade view of trade trajectory and volatility exposure across open horizons.

3. Agentic Autonomous Trading Tournament (/edge/agentic)

• Multi-Model Tournament: Evaluates 8 distinct LLMs acting as autonomous portfolio managers:
• Gemma 4 26B (gemma-4-26b-a4b-it)
• Qwen 3.8 27B (qwen3-8-27b)
• GPT-OSS 120B (gpt-oss-120b)
• GLM 5.3 (glm-5-3) & GLM 5.3 Flash (glm-5-3-flash)
• DeepSeek V4 Flash (deepseek-v4-flash)
• MiniMax M3 (minimax-m3)
• Llama 3.3 70B (llama-3.3-70b-instruct)
• *Baseline*: crowd-follow (no-LLM consensus control)
• Independent $10,000 Portfolios: Each model starts with a simulated $10,000 cash balance and autonomy to scan markets, evaluate orderbooks, and execute Immediate-or-Cancel (IOC) paper orders.
• Equity Curves & Cycle Health: Renders cumulative returns ($10,000 reinvested), drawdown metrics, trade logs, and hourly cycle execution health (tracking provider availability, token budgets, and trade triggers).

5-minute crypto markets

The local crypto micro-market model in src/analyzing_llm_rationale/crypto_5m.py is built for 5-minute UP/DOWN markets where the goal is profitable selective trading, not constant action. It combines:

• shrunken-drift lognormal moneyness pricing,
• AR(1) return forecasting with EWMA volatility,
• fixed or adaptive logistic ML features from momentum, reversal, volatility regime, range position, and volume imbalance.

Each forecast returns predicted_outcome, probability_up, component_probabilities, model-vs-market edge, and a fee-aware strategy. The strategy only recommends a trade when net expected value clears fees and the configured no-trade threshold.

.venv/bin/python scripts/crypto_5m_backtest.py \

--benchmark \ --symbols BTC,ETH,SOL \ --days 1 \ --max-candles 1600 \ --lookback-minutes 60 \ --horizon-minutes 5 \ --market-probability 0.50 \ --fee-bps 2 \ --ml-modes fixed,adaptive \ --edge-thresholds 0,0.01,0.03,0.05,0.08 \ --selection-fraction 0.6 \ --folds 4 \ --training-window 120 \ --max-rows 80 \ --benchmark-log data/crypto_5m_benchmark_runs.jsonl

Use fold_aggregate and evidence_quality before risking capital. If selection is unstable or holdout PnL is weak, the correct profitable action is to abstain. --benchmark-log appends a compact JSONL record for tracking whether the selected threshold and model mode keep working across benchmark runs. Resolve completed markets against Binance candles:

.venv/bin/python scripts/crypto_5m_backtest.py \

--resolve \ --symbol BTCUSDT \ --target-price 62400.52 \ --start-time-ms 1780000000000 \ --horizon-minutes 5 \ --predicted-outcome down

The resolver returns pending before expiry and resolved afterward with actual_outcome, resolved_price, and prediction_correct.

Record and resolve paper signals over time:

.venv/bin/python scripts/crypto_5m_backtest.py \

--paper-signal \ --symbol BTCUSDT \ --market-probability 0.50 \ --fee-bps 2 \ --signal-log data/crypto_5m_signal_log.jsonl

.venv/bin/python scripts/crypto_5m_backtest.py \ --resolve-signal-log \ --signal-log data/crypto_5m_signal_log.jsonl

.venv/bin/python scripts/crypto_5m_backtest.py \ --signal-summary \ --signal-log data/crypto_5m_signal_log.jsonl \ --min-resolved-trades 200 \ --min-total-pnl 0 \ --min-hit-rate 0.53

.venv/bin/python scripts/crypto_5m_backtest.py \ --paper-loop \ --symbols BTC,ETH,SOL \ --iterations 12 \ --sleep-seconds 60 \ --market-probability 0.50 \ --fee-bps 2 \ --signal-log data/crypto_5m_signal_log.jsonl

The signal log is the running dataset for model improvement: each record stores the forecast, recommendation, later actual_outcome, correctness, and pnl_per_contract for actual buy_up/buy_down paper trades. Use --signal-summary to audit whether resolved paper trades are positive after fees; trade_ready stays false until the configured trade count, PnL, and hit rate thresholds are met. Use --dry-run with --paper-loop to preview signals without writing the log.

Production Deployment Notes

Production is served from the custom domain:

https://foresea.ink

The Cloud Run service name, project ID, and region are set at deploy time