Skip to content

Pipeline manifests

Package: mas-lab-bench · Two formats — do not mix them in one file.

A pipeline is an ordered list of pipeline steps (e.g. extract_trace_stats, plotnine) that read run artifacts such as events.jsonl and write CSV/PNG under results/. Level hooks live in experiment.yaml (run.post, item.post, scenario.post, experiment post:). A standalone pipeline: file is run with mas-lab benchmark pipeline run.

Terms: glossary.md · Hands-on: Tutorial 3.

Post-processing pipeline (pipeline: root)

Schema: pipeline.schema.yaml · API id: pipeline-post

Used by the benchmark executor for analysis DAGs embedded in experiments or standalone files.

pipeline:
  name: t3-analysis
  output:
    base_dir: ./output
  steps:
    - name: extract
      type: extract_trajectories
      depends_on: []
    - name: plot
      type: plot_trajectory
      depends_on: [extract]

Steps reference upstream outputs via template paths ({{run.output_dir}}/...) in experiment context.

Level-scoped steps (scope: / in: / out:)

A step embedded inside an experiment manifest's run:/item:/scenario:/application: block is materialized once per folder at that level; its scope is inferred from the enclosing block (explicit scope: is only needed on a standalone pipeline file with no enclosing block). in:/out: name the artifact a step reads/writes — declared in that level's own artifacts: map. A step whose in: names an artifact from the level below fans in every child instance (config["artifact_paths"]); see experiment.md and multi-scenario-format.md.

For a standalone pipeline: file, the pipeline schema calls the per-item scope test; that is the internal scope name. Experiment manifests use the public level key item: (test: is a deprecated alias there).

run:
  artifacts:
    trace: { type: trace, path: "{run_dir}/traces/events.jsonl" }
    metrics: metrics
  post:
    - name: eval-quality
      type: eval_mce
      in: trace
      out: metrics

item:
  artifacts:
    df: { type: dataframe, path: "{level_dir}/data.csv" }
  post:
    - name: gather-test       # fans in every run's `df` under this test folder
      type: gather_level
      in: df
      out: df

Pipeline library (kind: Pipeline)

Schema: pipeline-manifest.schema.json · API id: pipeline

Stored under lab/pipelines/*.yaml; edited in mas-lab-ui PipelineBuilder; validated by POST .../pipelines/validate.

apiVersion: mas/v1
kind: Pipeline
metadata:
  name: analysis
spec:
  steps:
    - name: extract
      type: extract_trajectories
      depends_on: []
    - name: stats
      type: extract_trace_stats
      depends_on: [extract]
      config:
        output: "{output_dir}/results/trace_stats.csv"

UI may emit x-canvas-positions (stripped before execution).

Steps, types, and artifacts

  • DAG: depends_on with cycle detection (controller + executor).
  • Step types: registered processors (extract_trajectories, extract_trace_stats, eval_mce, plot_*, …). Catalog: pipeline steps.
  • Artifacts: typed values passed between steps — in-process during execution; serializable to files or infra sinks when a step writes outputs paths.
  • Cache: each step fingerprints its type, config, and upstream steps under <output_dir>/.cache/. Unchanged steps are skipped; a fingerprint change reruns that step and everything that depends_on it. Force with mas-lab benchmark pipeline run … --force STEP or mas-lab benchmark step restart.
  • Streaming: set streaming: true on a step. During a MAS run the runtime publishes transition dicts on mas.runtime.boundary.obs.event_stream.EventStream. The step's on_event is called for each; execute still writes artefacts afterward. Demo UIs subscribe to the same stream.

See also