Run Specs

Source mirrored from seocho/docs/RUN_SPECS.md

seocho run executes one end-to-end flow — index documents, ask questions, write a report — from a single YAML file. A run spec is the run-scoped layer on top of the existing design dialects:

run-scoped vocabulary (models, graph connection, inputs, output) lives here
agent behavior is delegated to agent design specs
ingestion behavior is delegated to indexing design specs

A run spec never redefines a key those dialects already own; it references or embeds their documents.

At A Glance

Use this page when you want a reproducible run instead of one-off Python code.

Reader task	Section to read	Output
create the first YAML file	Quickstart and Minimal spec	`seocho.run.yaml`
understand all supported fields	Full spec	a complete run config
choose ontology strictness	Ontology enforcement	`strict`, `guided`, or `open`
compare multiple variants	Templates and sweeps	one comparison report
debug a failed run	CLI and Report	preflight errors and `report.md`

The short mental model:

File	Job
`seocho.run.yaml`	one resolved run: index, query, report
`run.yaml.j2`	a template for multiple variants
`seocho.sweep.yaml`	the list of variants to render and compare

Running the CLI

SEOCHO standardizes on uv. From a repo checkout, prefix every command with uv run — uv resolves the project environment and syncs dependencies, so there is no venv to activate:

uv run seocho run …
uv run seocho sweep …

If you installed SEOCHO into your own environment instead (uv pip install seocho), drop the prefix and call seocho … directly. The examples below use the bare seocho form for brevity; prepend uv run when working from a clone.

Quickstart

uv run seocho run --init    # writes a commented seocho.run.yaml template
# edit ontology/documents/questions, then:
uv run seocho run           # or: uv run seocho run path/to/config.yaml

A working example ships in the repo:

export MARA_API_KEY=...
uv run seocho run examples/run/quickstart.yaml

Minimal spec

With no graph server and no model keys beyond one provider env var:

ontology: ./schema.yaml
documents: ./docs/
questions:
  - Which companies reported revenue growth?
  - Who is the CEO of Acme?

Defaults: mara/MiniMax-M2.5 for both phases, embedded LadybugDB (.seocho/local.lbug), guided enforcement, pipeline execution mode.

Full spec

The full spec has five practical blocks:

Block	Controls	Usually start with
`ontology`	schema source and enforcement policy	`path` + `enforcement: guided`
`documents`	files or records to index	`path` + `recursive: true`
`models`	model used for indexing and querying	`default`
`graph` / `vector`	storage backend	omit for embedded LadybugDB
`query` / `questions` / `output`	questions and report location	one or more `questions`

name: filings-demo                  # default: config filename stem
description: Optional free text.

ontology:
  path: ./schema.yaml               # YAML / JSON-LD / TTL (Ontology.load)
  enforcement: guided               # strict | guided | open (default: guided)

documents:
  path: ./docs/                     # .txt .md .csv .json .jsonl .pdf
  recursive: true

models:
  default: mara/MiniMax-M2.5        # provider/model for both phases
  indexing: mara/MiniMax-M2         # per-phase override → separate client
  query: mara/MiniMax-M2.5

graph: bolt://localhost:7687        # omit for embedded LadybugDB
graph_user: neo4j
graph_password: ${NEO4J_PASSWORD:-password}
database: neo4j                     # omit to derive from the ontology name
workspace_id: filings_demo          # default: derived from name

# graph also accepts a mapping with an explicit backend kind:
# graph:
#   kind: dozerdb                   # neo4j | dozerdb | ladybug
#   uri: bolt://localhost:7687      # ladybug uses `path:` instead
#   user: neo4j
#   password: ${NEO4J_PASSWORD}
#   database: mydb

vector:                             # optional hybrid-search vector store
  kind: faiss                       # faiss (in-memory) | lancedb (on-disk)
  embedding: fastembed              # local bge (default, no network) or an
                                    #   LLM provider preset (mara, openai, ...)
  # embedding_model: BAAI/bge-small-en-v1.5
  # uri: ./.lancedb                 # lancedb only
  # table_name: seocho_vectors      # lancedb only

indexing:
  design: ./indexing_design.yaml    # optional IndexingDesignSpec (path or inline)
  category: filing
  force: false

agent:
  design: ./agent_design.yaml       # optional AgentDesignSpec (path or inline)
  execution_mode: pipeline          # pipeline | agent | supervisor
  routing_policy: balanced          # fast | balanced | thorough

query:
  reasoning_mode: true
  repair_budget: 1
  answer_style: concise             # concise | evidence | table
  limit: 5

questions:                          # strings, or mappings with expectations
  - Which companies reported revenue growth?
  - question: Who is the CEO of Acme?
    expect: Jane Park               # recorded in the report, not auto-graded
    id: ceo-check

output:
  dir: runs                         # report lands in runs/<name>-<timestamp>/

Omitting questions entirely makes the run index-only.

Ontology enforcement

ontology.enforcement declares the admission policy for extracted graph data against the ontology vocabulary:

| Mode | Use it when | Write behavior | Failure behavior | | --- | --- | | strict | the graph must stay inside a closed vocabulary | reject invalid chunks | empty extraction is allowed; validation_on_fail defaults to reject | | guided | you want the default product behavior | write content and report validation errors | relaxed retry and Entity fallback remain available | | open | you are discovering a new ontology | write content and mark out-of-vocabulary facts | validation_on_fail: reject is incoherent |

Implementation notes:

strict disables relaxed retry and heuristic fallbacks.
strict validates dangling endpoints and domain/range conformance through the broader chain.
open stamps out-of-vocabulary nodes and relationships with _out_of_ontology: "true" for later governance review.
These modes are admission policies for extracted data, not CWA/OWA inference semantics. Query-time entailment is unchanged.

Domain-adaptive guardrail selection

Instead of a fixed ontology.path, a run can declare ontology.select to let SEOCHO pick the best guardrail ontology for the target corpus (ADR-0123). The runner scores each candidate against the corpus profile with the corpus-aware scorecard and chooses — richest-adequate for entity/qualitative corpora, leanest for numeric corpora (where vocabulary enrichment does not improve answers, ADR-0122):

ontology:
  select:
    candidates:
      lean: ./fibo_minus.jsonld
      rich: ./fibo_plus.jsonld
    corpus_profile: ./corpus_profile.json   # {label: freq} or build_corpus_profile output
  enforcement: guided

Candidates can instead be built automatically from a compiled FIBO catalog (ADR-0142): select.fibo.catalog points at catalog.json; each module is corpus-bridged (lexical + semantic) and the best is chosen. bridge: stable derives a multi-model auto seed (uses the MARA provider — ADR-0140); bridge: lexical stays offline. The chosen guardrail is held in memory (version-pinned to the FIBO commit), no file needed.

ontology:
  select:
    fibo:
      catalog: ./outputs/semantic_artifacts/fibo/latest/catalog.json
      modules: [BE, FBC, FND, SEC]   # omit → all modules
      bridge: stable                 # stable (multi-model auto seed) | lexical (offline)
      derive_models: [DeepSeek-V3.1, MiniMax-M2.5, gpt-oss-120b]
    corpus_profile: ./corpus_profile.json
  enforcement: guided

The corpus profile comes from an open (ontology-free) extraction over the corpus (seocho.ontology_scorecard.build_corpus_profile). At run time the chosen ontology is logged ([guardrail] selected '<name>' …) and recorded on the run. Provide either ontology.path (fixed) or ontology.select — not neither.

The enforcement policy is compiled by seocho.EnforcementPolicy from AgentConfig.ontology_enforcement; agent design specs may declare ontology.enforcement too, and an explicit run-spec value overrides the design (the implicit guided default never does). These are admission policies for extracted data — not CWA/OWA inference semantics; query-time entailment is unchanged in every mode.

Storage backends

The graph store is selected by graph.kind (or inferred from a bare string: bolt-scheme URI → Neo4j/DozerDB, anything else → embedded LadybugDB path). The optional vector: section adds a hybrid-search vector store (Seocho(vector_store=...) / search_similar()):

Backend	Key	Notes
LadybugDB (embedded)	`graph.kind: ladybug` / blank	zero-infra default; one file = one graph
Neo4j / DozerDB	`graph.kind: neo4j\|dozerdb` + `uri:`	per-`database` isolation on one server
FAISS (vector)	`vector.kind: faiss`	in-memory, rebuilt per run
LanceDB (vector)	`vector.kind: lancedb` + `uri:`	on-disk; sweeps isolate the uri per variant

vector.embedding defaults to fastembed (local bge, no network, 384-dim auto-derived); set it to a provider preset (mara, openai, …) for HTTP embeddings. RDBMS backends are not supported — the SDK’s GraphStore contract has no relational implementation today.

In sweeps, on-disk vector stores get the same isolation as embedded graphs (blank lancedb uri → <sweep>/<variant>/vectors.lancedb), and a bolt graph shared by multiple variants prints a note that isolation rides on per-variant database/workspace only.

Per-phase models

When models.indexing and models.query differ, the runner builds two clients sharing one graph store, ontology, and workspace — per-phase model separation without per-call plumbing. Env-driven routing (SEOCHO_MODEL_ROUTING) still applies within each phase if configured.

Environment variables in values

Any string value may interpolate ${VAR} or ${VAR:-default}. An unset variable without a default is a config error. Put env var names in YAML, never literal API keys; provider keys are read from the provider’s standard env var (MARA_API_KEY, OPENAI_API_KEY, …).

CLI

seocho run [CONFIG] [options]

  CONFIG            Run spec YAML, or a Jinja2 template (*.yaml.j2)
  --init            Write a commented template and exit (refuses to overwrite)
  --dry-run         Validate config + offline preflight; no LLM calls
  --only index|query  Run a single phase (query reuses the existing graph)
  --var KEY=VALUE   Template variable (*.j2 only; repeatable, dotted keys)
  --vars FILE       YAML file of template variables (repeatable)
  --show-rendered   Print the rendered YAML (pre-${ENV}) and exit
  -o, --output DIR  Report directory (default: runs/<name>-<timestamp>/)
  --force           Re-index files even if unchanged
  --output-json     Machine-readable output

Exit codes: 0 ok · 1 runtime/preflight failure · 2 invalid config

Every run starts with a preflight that reports all failing checks (ontology loads, documents found, API key present, graph reachable) before anything spends tokens. --dry-run is the same preflight without the graph connection attempt.

Report

Each run writes report.json (machine-readable: run metadata, per-file indexing stats, per-question records with latency and errors) and report.md (human summary) under output.dir/<name>-<timestamp>/. Empty answers and per-question errors are surfaced in the summary table; a question error marks the run as failed (exit 1) without aborting remaining questions.

Templates and sweeps

Doctrine:

run — one resolved spec, one report. A spec may be a Jinja2 template; rendering produces the one spec.
sweep — one template × N variable sets → N runs → one comparison summary.
experiment — extraction-only micro-benchmark over a single text/dir (no query phase, no run reports).

Two substitution layers

Layer	Syntax	Resolved	Use for
Jinja2	`{{ var }}`, `{% for %}`	authoring/render time, before YAML parse	parameters, variants, structure
Env	`${VAR}`, `${VAR:-default}`	load time, after rendering	secrets — never persisted into artifacts

Rendering is decided by file extension only (*.j2); plain .yaml configs never touch the template layer, so question text containing {{ stays untouched (use {% raw %} for literal braces inside templates). Supplying --var/--vars for a non-template config is an error.

Authoring rule: quote every string substitution ("{{ model }}"), never quote numeric/boolean ones ({{ limit | default(5) }}) — this avoids YAML flow-mapping breakage and the Norway problem in one rule.

Variable precedence (low → high): template | default(...) < sweep vars: < variant vars: < --vars files (in order) < --var flags. Mappings deep-merge; scalars and lists are replaced. --var keys may be dotted (models.indexing=...) and values are YAML-parsed (limit=10 → int). variant and sweep are reserved names — seocho sweep injects variant.name, variant.index, and sweep.name per variant.

Sweep file

name: enforcement-shootout      # default: filename stem
template: ./run.yaml.j2         # required; rendered once per variant
vars:                           # shared by every variant (optional)
  model: mara/MiniMax-M2.5
variants:                       # required, non-empty; unique names
  - name: guided
    vars: { enforcement: guided }
  - name: strict
    vars: { enforcement: strict }
output:
  dir: runs

seocho sweep [SWEEP] [options]

  SWEEP                  Sweep spec YAML (default: ./seocho.sweep.yaml)
  --init                 Write seocho.sweep.yaml + run.yaml.j2 and exit
  --dry-run              Render + validate every variant + offline preflight
  --show-rendered [NAME] Print rendered YAML (one variant, or all) and exit
  --only-variant NAME    Run a subset (repeatable)
  --var / --vars         Variable overrides applied to ALL variants
  --fail-fast            Stop at the first failed variant (default: keep going)
  -o, --output DIR       Sweep root (default: runs/<name>-<timestamp>/)

Exit codes: 0 all variants ok · 1 any variant failed · 2 config error

Config errors are collected across all variants up front and nothing runs (exit 2); runtime failures keep going by default and the comparison table marks them (exit 1).

Variant isolation

Each variant is fully isolated, automatically:

blank graph: → its own embedded store at <sweep>/<variant>/graph.lbug (one .lbug file is one graph — paths isolate, names do not);
bolt:// targets keep the URI but get a per-variant database when blank;
workspace_id is always suffixed with the variant name — the response cache is keyed by workspace, so two variants differing only by model would otherwise serve each other’s cached answers;
.seocho_index change tracking is disabled (track=False), so variant N+1 never skips files as “unchanged” and your docs directory stays clean.

Variants run sequentially: the embedded store is a single-writer engine and parallel variants would mostly time-slice the same LLM rate limit.

Sweep artifacts

runs/<sweep-name>-<timestamp>/
  summary.json / summary.md     # comparison table + per-variant status
  <variant>/
    rendered.yaml               # resolved spec (pre-${ENV}, paths absolutized)
                                #   reproduce standalone: seocho run rendered.yaml
    report.json / report.md
    graph.lbug

A runnable example lives at examples/run/sweep-enforcement/ — the quickstart documents indexed under guided, strict, and open enforcement, compared in one table.