AUTONOMOUS ML PIPELINE · LIVE

Drop a dataset.Get a modelyou can trust.

Your junior ML engineer that never sleeps. It profiles the data, hunts for leakage, trains and races models, reviews its own work, and hands you a report — in about a minute.

3 real OpenML datasets · downloadable · no signup
≈ FLAMLmatches an AutoML baseline on 10 unseen benchmarks (0.987 of FLAML, 95% CI 0.949 to 1.011)
0 unsafeactions executed in every prompt-injection test (0/49 real-LLM runs)
683tests · 91.8% coverage
~52 sdataset to report, on a free CPU (measured median)

Sample datasets

Three real tables. Pick one.

Public OpenML datasets bundled with this demo. The agent uses the scripted planner (no LLM) and stops at a hard time limit; one run executes at a time, so you may wait in a short queue.

How it works

Five steps, no hand-holding.

01 PROFILE

Look at the data

Rows, columns, types, missing values, duplicates.

02 LEAKAGE

Check for cheats

Columns that give the answer away are flagged and dropped. Skipped in this demo to fit the free CPU.

03 RACE

Train and compare

A baseline, a linear model, boosting and a forest, scored on held-out data.

04 REVIEW

Check its own work

A deterministic reviewer signs off or sends the run back.

05 EXPLAIN

Write it up

A plain-English summary, the top features and a model card.

Not in this demo

What the full system does

  • An LLM planner and a critic, with a structural guard against prompt injection.
  • Leakage detectors and FLAML AutoML (both switched off here: this server has 0.1 of a CPU).
  • Agent-written feature code in a sandbox (Docker or E2B), and gated model deployment with a single-use approval.
  • Bring-your-own CSV. The demo has no upload at all: the server accepts no files.

Benchmarks

Agent benchmark v5 (headline; unseen partition, each arm run once)

SCRIPTED planner (no LLM)
0.987 [0.949, 1.011]
pass vs linear: 7/10 (Wilson 0.40-0.89)
REAL-LLM planner v2
0.989 [0.950, 1.012]
pass vs linear: 8/10 (Wilson 0.49-0.94)
ArmNormalized score [95% CI]Paired differencesPass vs linearCost / latencySource
SCRIPTED planner (no LLM)0.987 [0.949, 1.011]vs linear 0.189 [0.075, 0.315]; vs FLAML -0.013 [-0.051, 0.011]7/10 (Wilson 0.40-0.89)latency p50 125s / p95 199s per task; 0 LLM requests, 0 tokens, $0agent_v5_scripted.json
REAL-LLM planner v20.989 [0.950, 1.012]vs linear 0.190 [0.076, 0.316]; vs FLAML -0.011 [-0.050, 0.012]8/10 (Wilson 0.49-0.94)latency p50 135s / p95 226s per task; 130 LLM requests, 19562 tokens, $0agent_v5_llm.json

Linear baseline normalized mean 0.798 [0.680, 0.908] (n=10); FLAML = 1 by definition. Source: baselines_v5.json. Paired LLM minus scripted: 0.001 [-0.001, 0.004] (n=10).

Figures are read from docs/METRICS.md when this page is built (file hash 997814d2f3e3, built 2026-11-15 10:58 UTC). A demo run uses a shorter budget and does not reproduce them.

Verified metrics: everything else

Generated by scripts/render_metrics.py. Each row names its source file or CI run. Planner and LLM mode are stated in every agent row.

Leakage detector

  • eval4 (unseen planted partition, 30 variants): recall 0.967 (bootstrap 95% [0.9, 1.0]), clean-twin false alarms 0.000 (Wilson 95% [0.0, 0.1135]). Source: leakage_v3.json.

Injection defence

  • Real LLM in the loop (injection_llm_v2): attacker action proposed 0/49, unsafe actions executed 0/49. Source: injection_llm_v2.json.
  • Evasion DEV (injection_llm_v3.json; burned, the scanner was tuned on it): attacker actions 0/8, scanner flagged 8/8.
  • FRESH 14-payload held-out evasion set (hash-committed before the run, run once): attacker actions proposed 0/14 (Wilson 95% upper bound 0.215), unsafe executed 0, 1 non-train/finish proposal (a benign read_file train.csv, guard allowed). The scanner flagged only 1/14 of the fresh payloads, so the model's own behaviour and the structural guard carry this result.
  • 5 full agent runs with injected column names (real planner v2 + critic v3): 5 finished, unsafe tools executed 0, payload reached the critic 0 times (by design the planner never sees column names, so this is a null-exposure check, not an attack); critic flagged 3/5 for ordinary quality reasons (rules flagged 2). Source: injected_runs_v1.json.

Critic (advisory)

  • v5 runs, critic v2 vs rule reviewer: agreement 18/20 (Wilson 95% 0.70-0.97); confusion {'rule=APPROVE|critic=APPROVE': 18, 'rule=REQUEST_CHANGES|critic=APPROVE': 2}; 14 requests. Source: critic_v5.json.
  • Dev calibration (mutated states): see critic_calibration.json.
  • FRESH planted set (critic v3, schema-enforced answers to failed rules): recall 10/15 (Wilson 95% 0.42-0.85); false alarms 1/19 clean runs (Wilson 95% 0.01-0.25); 4 clean items not run (40-request cap). By type: {'cv_instability': '1/2', 'gap_beyond_family_range': '2/3', 'partial_leakage_check': '2/2', 'rule_fail_baseline_margin': '0/3', 'tiny_test': '2/2', 'unresolved_leakage': '3/3'}. Source: critic_v6.json, set benchmarks/critic_set_v6.json (sha256 6595774668473af9, committed before the first call).
  • The v5 agreement above (18/20) is burned and post-hoc: the critic was tuned after reading v5-era failures.
  • FRESH planted set v2 (53 items, hash bfdcaf09a5aa0346, failing-margin items now unambiguous; a DIFFERENT set from v1, not paired): completed run recall 18/23 (Wilson 95% 0.58-0.90), false alarms 0/30 (0.00-0.11). First pass (8 items hit the gateway token budget) was 15/20 and 0/25; the 8 were completed later with the same prompt (labelled completion). Failing-margin items 0/4: the model dissents with numbers, a documented weakness. Source: critic_v2_eval.json.

E2B sandbox backend

  • 20/20 sequential execs (default template, internet off): total p50 1.87 s / p95 2.67 s (create p50 0.38 s, upload 0.56 s, exec 0.34 s, download 0.55 s); n=20, one machine, an order-of-magnitude figure. Live suite: 8 passed, 0 failed, 0 xfailed, 2 skipped (the 2 skips need a custom template with pyarrow, not built). Source: e2b_v1.json.

Demo

  • LIVE (host: Render free plan, 512 MB, 0.1 vCPU): https://junior-ml-dev-demo.onrender.com . Verified: health 200, landing 200, admin without auth 401, upload attempt 403, deploy path 404, reports [200, 200, 200]. Sample run (diabetes) p50 52.0 s over 3 runs ([52.8, 52.0, 51.9]). Cold start after >15 min idle: health 32.7 s after >15 min idle; first sample run 105.8 s end to end (includes ~45-60 s warm-worker boot); status done. Local peak process-tree RSS of a sample run: 309-327 MB local (Windows psutil process tree, FLAML 15 s); web process alone 63 MB idle. Source: demo_render_v1.json.
  • Cut to fit the free plan: FLAML off; leakage detectors skipped (recorded as a PARTIAL check); hard wall 60 s, soft wall 35 s; web process no longer imports the full API (225 MB -> 63 MB); a warm fork server loads pandas/scikit-learn/lightgbm once (they took 44 s per run on the 0.1 vCPU).
  • Per-sample run time (s): credit-g 37.8; diabetes 52.0; wine-quality 27.4 (live, submit to done: diabetes median of 3 runs, the others one steady-state run; scripted planner, FLAML off, leakage detectors skipped; landing page 35.4 KB of HTML, CSS and JS, fonts excluded).
  • Static precomputed site (no code runs there): https://giri-harsh.github.io/junior-ml-dev-demo/ .

Quality gate

  • Verified via local quality gate: ruff, format, mypy, pytest — 683 tests, 91.8% coverage (floor 91%).

Load test

UsersRPSp50 msp95 msp99 mserror rate
536.21402102400.000
2575.126150016000.000
10065.7140670096000.178

Source: load_v1.json (CI run 40945724068); hosted-runner numbers, not a capacity claim.

Pick a dataset

These are real public datasets from OpenML. Download any of them to check exactly what the agent sees.

Upload your own CSV Coming soon

Bring-your-own-data is in the works. For now, try one of the datasets above.