Reproducible multi-agent experiments,
from hypothesis to paper-ready results

Research infrastructure
shouldn't be your research

Multi-agent experiments require orchestration, evaluation, reproducibility, and statistical analysis. Most researchers build this from scratch for every paper — then throw it away.

Everything between
hypothesis and publication

agent = Agent(
  strategy="debate",  # swap to compare
  max_iterations=6,
)

grid = ExperimentGrid(
  conditions={
    "strategy": ["react", "debate"],
  },
  seeds=[42, 123, 456],
)
results = await grid.run()

results.to_latex("table1.tex")
results.to_csv("results.csv")
results.compare(A, B)  # p-value

$ jamjet replay exec_abc
$ jamjet fork exec_abc \
  --override-input '{"model":"gemini"}'

# workflow.yaml
check:
  type: eval
  on_fail: retry_with_feedback
  max_retries: 2

$ jamjet init my-study \
  --template research
# agents/ baselines/ experiments/
# evals/ results/ workflow.yaml

Most agent frameworks prioritize apps
over experimental reproducibility

From hypothesis to Methods section

One research afternoon, end to end

grid = ExperimentGrid(
  conditions={"strategy": ["react", "plan_and_execute",
    "critic", "reflection", "consensus", "debate"]},
  seeds=[42, 123, 456],
)
results = await grid.run()

results.to_latex("table1.tex", caption="Strategy comparison")
# Outputs booktabs table with mean +/- std per condition

$ jamjet replay exec_debate_seed42
# Restores from checkpoint. Saves tokens + cost.

results.compare("debate", "react")
# => {p_value: 0.023, effect_size: 0.41, significant: true}

$ jamjet fork exec_debate_seed42 \
  --override-input '{"model":"gpt-4o"}'
# Same execution, different model. Instant ablation.

Start as a simple Python agent, scale into reproducible experiment runs — without rewriting your stack. See the quickstart →

What a result looks like

Task: summarize a 2,000-word policy document. 6 strategies, 3 seeds each. Scored by LLM-judge (0–1). Local Ollama, Llama 3.

debate vs. react: p = 0.012 (Welch's t-test, n = 3 seeds). This table was generated by results.to_latex("table1.tex") — zero manual formatting.

Illustrative results from internal testing. Your numbers will vary by model, task, and hardware.

Why not just scripts?

Custom scripts work for one-off experiments. They break down when you need to reproduce, compare, or build on prior work.

Patterns from published research

Built for how you work

Start your experiment

From pip install to running multi-agent experiments in under 5 minutes.