Monte-Neo strategy verifier

The independent verifier for trading strategies written by AI agents and humans.

Catch look-ahead bias, hidden trading costs and overfitting before a backtest reaches your money.

Docs · Quick start · Use from agents · Verifier API · Trap Suite · Changelog

Did it catch a leak in your backtest? Star the repository so other quants and agent builders can find it, or tell us what it missed .

---

The problem

A coding agent can turn a trading idea into a backtest in minutes. It will then tell you the strategy returns 40% a year with a Sharpe of 3. Most of the time that number is wrong, for the same few reasons:

• Look-ahead bias. The code reads future bars: shift(-1), centred windows, bfill, statistics over the whole series, np.gradient, an FFT filter.
• Missing costs. The edge is smaller than fees and slippage, or it disappears when the fill comes one bar later.
• Selection bias. The agent tried 300 variants and reports the best one as if it were the only one.

Backtest libraries run whatever code you give them. None of them tell you the backtest itself is broken.

The solution

Monte-Neo checks the backtest, not the idea. Give it the price data and the strategy code or its positions. It returns one of four verdicts, the checks behind the verdict, concrete next steps and a reproducible, optionally signed certificate.

The agent's strategy used shift(-1), so it knew the next close. Monte-Neo found the leak in four independent ways and named the line. After the fix, no look-ahead is left, and the verifier tells the truth: on a random walk, the strategy has no edge after costs.

The same run as a report for people (--html report.html: one file, no scripts, no network):

More reports, including an honest strategy and a parameter search, are in the gallery.

Text output of the first run

$ monte-neo verify --ohlcv prices.csv --strategy agent_strategy.py --n-trials 40

REJECT certificate 7f0603b0e604bea5 check category status summary data_integrity integrity pass OHLCV is clean lookahead_truncation lookahead fail truncation probe: LEAK DETECTED lookahead_perturbation lookahead fail future-perturbation probe: LEAK DETECTED lookahead_static_lint lookahead fail static lint: negative_shift implausible_accuracy lookahead fail next-bar hit rate 1.000 over 2999 bars (z 54.8) net_profitability economics fail net total return -64.97% after costs deflated_sharpe statistics fail deflated Sharpe 0.000 over 40 trial(s) ... (16 more checks) → The signal at bar t changes when later bars are removed: compute features only from rows <= t (no shift(-k), centered windows, bfill or full-sample stats). → Fix the flagged source lines (negative shift, center=True, backward fill) and re-run verify. Lines: 6.

The run used a synthetic random walk; output shortened.

New in v0.53: from a rejected backtest to a committee

Repair the strategy, without overfitting it

monte-neo repair --demo

monte-neo repair --ohlcv data.csv --strategy my_strategy.py --out repair/

Optimisers maximise the return, so they overfit. repair minimises defects instead. It first corrects look-ahead and repainting. Then it picks from a short list of declared repairs (smoothing, holding period, no-trade band, tranches, neutralisation, caps, volatility target) on the first 70 % of the history, by the number of defects left and never by return, and counts every candidate as a trial. The last 30 % is a lockbox, opened once.

There are four outcomes:

• REPAIRED: on data the choice never saw, the strategy still earns after costs and is not rejected (the lockbox can still ask for more evidence, such as more trades).
• REPAIRED_NO_EDGE: the defects are gone and so is the profit, because the profit was in the defect.
• NOT_REPAIRABLE: what you have to do yourself.
• NOTHING_TO_REPAIR: there was nothing in the catalogue to fix.

The original file is never touched. On a random walk, none of the 68 leaking strategies of the Trap Suite comes out as REPAIRED. Guide.

Is it alpha? An Alpha Readiness Level for the committee

monte-neo alpha --demo --aum 2e5

monte-neo alpha --ohlcv universe.csv --scores scores.csv --aum 5e7 --book book.csv --n-trials 40

A ranking signal gets the checks a fund's research committee runs before it gives capital:

• signal power: rank IC, decay, quantiles, placebo scores;
• attribution: is it alpha, or momentum and short volatility under a new name?
• value to your book: the marginal Sharpe;
• implementability: square-root impact at your capital, capacity, short leg, rebalance day;
• robustness: subsets, start dates, bootstrapped markets;
• data and process: today's index members, mismatched exchange closes, decay after publication;
• multiple testing: the t ≥ 3 hurdle, the minimum backtest length;
• risk: named crises, tails.

Everything ends in one line, for example ARL 6/9 — blocked at 7: tradable and robust (rebalance_timing_luck). Missing evidence is "not assessed", never passed. Guide.

Evidence a model-risk committee can file

monte-neo dossier --certificate cert.json --framework sr11-7 --name "Momentum 20" --owner "Desk A"

monte-neo monitor --certificate cert.json --live live.csv # after launch: is the alpha dead?

A validation dossier is one folder with:

• one printable page: summary, checks, readiness level, assumptions and limitations, how to reproduce, sign-off fields;
• a clause map to SR 11-7, PRA SS1/23, MiFID II RTS 6, a crypto checklist or an academic appendix;
• the certificate and the environment.

The map shows evidence, never compliance. `monitor` compares live returns with the backtest, measures realised slippage and raises sequential alarms when the edge dies. Guide.

Machine learning, LLM signals, events and track records

• `fit` / `predict`. Define fit(train) and predict(model, df) instead of signal(df). The verifier refits only on the past, and ml_refit_gap rejects a model whose profit lived in fitting on the whole file.
• `--model-cutoff 2024-06`. An edge that exists only before a language model's training cutoff is look-ahead through the model's memory.
• `--events events.csv`. News, earnings or LLM outputs are joined by publication time. The report shows what a join by event time would have earned.
• `verify --returns manager.csv`. Due diligence from returns alone: smoothing, missing small losses, implausible consistency, significance given the manager's number of funds.
• `verify --legs legs.csv` and `--null-test pairs.py:pnl`. Phantom profit in pairs and stat-arb strategies: a rolling hedge ratio booked as position times the change of the spread earns a profit that no trader could, even on independent random walks. --legs re-derives the P&L from the quantity held in each leg and rejects a reported return the legs never earned (and writes the corrected series); --null-test runs the strategy on random walks, where it must earn nothing. Try monte-neo verify --demo-phantom.
• Your institution's rules and history. --policy policy.toml applies your thresholds and records them in the certificate. --trials-from mlflow:./mlruns (Optuna, W&B, Ray Tune too) counts the runs you already logged. --ledger on a shared drive counts the whole team's attempts.
• Where you already work. %%verify in Jupyter, Polars frames as input, and monte-neo dossier --lite for a one-page plain-language summary for people who do not read Sharpe ratios.
• Two more lint rules for ML code. full_sample_transform (a scaler or PCA fitted on all rows) and `target