Workshop · floor model for Survey of Software 1.200

muster

Orchestration is solved. Knowing whether you need it is not.

The job. You have a folder of a thousand vacation photos. You want them on a map — each pin where the photo was taken, each one carrying a tag and a caption saying what it is. Nothing exotic. A weekend project, or an afternoon at work.

Per photo, that is four pieces of work:

the workwhere it runswhat it costs
GPS from EXIF — where the photo was takenyour machinefree
Thumbnail — what the map pin showsyour machinefree
Vision encoding — shrink a copy small enough to sendyour machinefree, and it exists to cut the bill
The vision call — the tag and the captionsomebody's servermoney, every photo

Three of those run in this tab for real, on your own files. The fourth is the one that costs money, so this page prices it and never makes it — no key is collected, asked for, or accepted, and no photo leaves your machine.

Now: you are 190 photos in and the process dies. That is the whole of this survey, and the button below really does kill it.

Plain language. Switch to Engineering for versions, token counts and the measurement behind each claim.

The whole answer, first

Four numbers. The first two are the decision this survey is about; the second two are what the pipeline itself teaches once it is running on your files.

CPython 3.14 under Pyodide, Pillow 12.2. Panels 2–4 measure worker lifetime and IndexedDB durability in this tab; panel 1 measures your photos' dimensions and bytes and prices the vision call from published rates. Everything orange came from research rather than from your machine.

lost to one deploy

$4.73

what an in-memory run re-spends when it dies 190 photos into the thousand. The checkpointed lane re-spends one photo. Press Deploy in panel 2 and watch the counter move.

to retrofit it

rewrite

resumption is not a feature you bolt on. It is a property of holding run state somewhere other than your call stack, so deciding it late means deciding it twice. Every other difference in this category you can swap in a fortnight.

downscale saves

43%

on a high-resolution-tier model — and nothing at all on a standard-tier one, where the server downscales anyway. Same code, same photo. Panel 1 runs it on yours.

frameworks benchmarked

0

this page does not race LangGraph against Haystack and could not do so in a browser without faking it. It demonstrates the property they differ on, which is the part that decides.

Ask what happens when the process dies. If the answer is "we retry the whole job and that's fine", most of this category is solving a problem you do not have — a provider SDK and a validation library will serve. If a lost run costs money or trust, the field narrows to two options and the rest of the comparison stops mattering. The survey works the full decision; this page makes you feel the half of it that people skip.

Press start below to download CPython 3.14 (WebAssembly). First time only, then cached.

1. Your photos, through the pipeline

Drop in some photos — from your phone, from a folder, anything. The three local jobs run here for real: the GPS comes out of the EXIF, a thumbnail is made, and a shrunken copy is encoded for the vision model. Then the page prices the vision call it is not going to make, on your photos, at each engine's published rate.

The three local steps are extract_gps, make_thumbnail and encode_for_vision from the pipeline's image layer, vendored into core.py with their docstrings, changed only in that bytes arrive instead of a Path. The vision call is priced with Anthropic's published resize rule — ⌈w/28⌉ × ⌈h/28⌉ visual tokens, capped per tier — and the pipeline's own cost table.

…or drop your own photos here (they never leave the tab)

2. The kill

Two lanes now run the same job over the same photos. They differ in one thing: where the run keeps its state. Lane A keeps it in the process. Lane B writes a checkpoint to the browser's database after each photo. Start them, wait a few seconds, then press Deploy. The button really does kill both — it is not an animation.

Each lane is a Worker with its own Pyodide instance. Deploy calls worker.terminate() on both: no unwind, no finally, the WebAssembly heap goes with it. Lane B's checkpoint is an IndexedDB write made from inside the worker after each photo, so the main thread is not quietly holding the state that the kill is supposed to destroy. On restart each lane reports only what it can actually recover.

Lane A — state in the process

a plain script, or a framework that holds the run in memory

spent · re-spent

$0.00

idle

Lane B — state outside the process

a durable runtime: LangGraph's checkpointer, Temporal, Inngest

spent · re-spent

$0.00

idle

The counter that matters is money re-spent, not time. Time you lose once and forget. A re-spend appears on an invoice, and it is the number the person who signs it will ask about.

3. The human pause

Now put an approval step in the middle: at photo 7, both lanes stop and wait for a person. Lane A has to hold a live process open for however long that takes. Lane B wrote a checkpoint and stopped existing. Reload the page while they are waiting — Ctrl-R, or Cmd-R on a Mac. The page comes back in about two seconds with your photos still there and one lane still holding its place; that single keystroke settles the argument better than any paragraph.

Approval is the same mechanism as the crash: Lane A's pending state lives only in the worker, so a reload takes it — the workers are torn down with the document; Lane B's pending state is a row in IndexedDB keyed by job id, which survives a reload, a browser restart and a machine reboot alike. This is what "human-in-the-loop" costs architecturally, and it is why it appears in the same sentence as durability in every framework that offers it.

Closing the tab entirely, or quitting the browser, does the same thing — a reload is just the quickest way to prove it.

4. What durability costs

The demonstrations that stop here are selling you something. Keeping state outside the process has a price: every value in it has to be storable, which rules out the things programmers reach for most naturally — an open file, a socket, a callback, a class instance with a method on it. Tick the box and watch Lane B fail to save.

The checkpoint is a structuredClone into IndexedDB. Adding a function to the state object raises a real DataCloneError from the browser, not a simulated one. In production the equivalent is a serializer rejecting your state, a schema migration on the checkpoint table, and a rule that your run state is data rather than objects.

Untick: the checkpoint saves. Tick: it does not.

5. Who decides the next step — a quiz

Six flows, two buttons each. Pick one and the page tells you straight away whether you were right — and which survey answers that kind of question. One question separates this survey from its neighbour, and it is the cheapest way to find out which one you should be reading: does your code decide what happens next, or does the model?

The boundary test is the survey's organizing claim. Your code decides → 1.200, this page. The model decides → 1.201, agent frameworks. Without the test the two surveys collide, and the previous pass of 1.200 ended by recommending a multi-agent system, which is 1.201's territory.

Nothing answered yet — pick a button on any row above.

6. The vision call, without a key

If you want the tags and captions for real, do it yourself: take the prompt below to the assistant you already pay for, drag in the shrunken copies the page made in panel 1, and paste the answer back. No key exists anywhere in this — not collected, not asked for, not accepted — and the photos go from your machine to your assistant without passing through anything of ours.

The outbound leg is a prompt by vendor prefill URL; the return leg is a clipboard envelope, photo_analysis version 1, validated against the PhotoAnalysis contract the pipeline already publishes. Parsing it involves no model call, which is what makes the pattern cheap enough to be the default rather than a fallback. Per survey 2.078 the clipboard envelope is the only mechanism that closes both legs with zero setup, and the only one with no specification at any level.

The prompt, built from the photos you loaded:

What each option is for

ShapeWhat it is forThe catch
Durable workflow graph
LangGraph
A run must survive a restart, or pause for a human. The largest exit cost here, because it does the most.
General durable engine
Temporal, Inngest
The same property, for a business process that happens to contain model calls. More infrastructure; keeps the AI dependency shallow, which is often the point.
Declarative pipeline
Haystack 3.0
A document pipeline you can read as a diagram, on-premise or in a VPC. Not durable. Best migration story in the category.
Typed thin layer
PydanticAI
Ordinary code that needs validated structured output. Does not orchestrate. That is the feature.
Build-time compiler
DSPy + GEPA
The bottleneck is answer quality, not plumbing. Needs a metric that scores "better" automatically. Composes with any of the above, or none.
Nothing
provider SDK + MCP
A lost run costs nothing, or costs less than the framework does. None, until the day the answer to panel 2 changes.

Versions, verdicts and the six personas — three of which should adopt nothing — are in the survey. This page deliberately names no winner: the whole finding is that the winner depends on an answer only you have.

Why read the whole category

Nothing on this page is about artificial intelligence. A job made of steps, each one costing money, running on a machine that can stop between any two of them — that is payment capture, batch billing, video encoding, a payroll run, and the shipping label that got bought twice because someone refreshed. Every one of those industries worked out its answer decades ago and gave it a name: a transaction, a saga, a durable execution. LLM pipelines are the version where each step is expensive enough, and slow enough, that a beginner notices the problem in week one instead of year three.

Which is the useful thing about a young category. The mistakes are still visible. In a mature one they have been absorbed into a framework's defaults and nobody remembers what they were for — and that is the case for reading a survey of a whole field rather than a comparison of the two products someone happened to blog about.

muster is a floor model for Survey of Software 1.200 — LLM Orchestration Frameworks, built 2026-08-21. The three local jobs are working Python from a real photo-ingestion pipeline, running under Pyodide (CPython 3.14 compiled to WebAssembly) in this tab; nothing is uploaded and no API key is collected. The vision call is priced, never made — figures marked orange came from published rates rather than from your machine. This page benchmarks no framework.

Made by Ivan Schneider · Model Citizen Developer