LLM Orchestration: a domain explainer#

For a reader who is competent in software but new to this corner of it. No prior AI knowledge assumed.

The problem in one paragraph#

Calling a language model is an HTTP request. You send text, you get text. If that were all anyone needed, this category would not exist. What creates the need is that useful work usually takes several calls, arranged: fetch some documents, ask the model to pick the relevant ones, ask it to draft a summary, check the draft against a rule, ask a human to approve it, then write the result somewhere. Orchestration is the business of running that sequence — and of surviving the many ways it goes wrong.

The one distinction that organises everything#

There are two very different ways to run a sequence of model calls, and confusing them is the single most common source of bad advice in this field.

Your code decides the next step. You wrote the sequence down. The model fills in the steps — extracting, summarising, classifying — but the path is yours. This is a program that calls a model. It is testable in the way ordinary programs are testable: same input, same path.

The model decides the next step. You give it tools and a goal, and it chooses what to do next, possibly for many turns. This is an agent. It is powerful and it is non-deterministic: the same input may take a different path on Tuesday.

This survey (1.200) is about the first. Agents are 1.201. Both are legitimate; they have different failure modes, different testing strategies, and different costs, and a tool built for one is usually mediocre at the other.

Where the difficulty actually lives#

Newcomers expect the hard part to be prompting. It is not. The hard parts are:

Things fail in the middle. A provider returns a 529, a network blips, your process is restarted by a deploy. If step 9 of 12 dies, what happens? Do you re-run steps 1–8 — paying for them again, and hoping none of them sent an email? This is the question that separates serious tools from convenient ones, and it has a name: durability.

Output does not match its shape. You asked for JSON with four fields. You got JSON with three fields and a friendly sentence in front. Every production system needs a validation layer and a retry, and this is the most common LLM bug there is.

Waiting for humans. Real workflows pause for approval. A pause can last a day, which means the state of the run has to live somewhere other than the memory of a process.

Quality is a moving target. The sequence runs correctly and the answers are mediocre. This is a different problem from all of the above, needs a different tool, and is consistently mis-filed with them.

The vocabulary#

Chain / pipeline. A fixed sequence of steps. Output of one feeds the next.

Graph / workflow. A sequence with branches and loops, declared as nodes and edges. The runtime walks it. This has become the category’s shared model — several vendors arrived at it independently.

State. What a run knows so far, carried between steps. Whether this lives in memory or in a database is the durability question.

Checkpoint. A saved copy of state, so a run can resume after a crash rather than start again.

Human-in-the-loop. A deliberate pause for a person to review, edit, or approve.

Tool / function calling. Letting the model invoke something — a search, a database query, an API. The model asks; your code performs.

MCP (Model Context Protocol). A standard way to expose tools so that any client can call them. Before it, every framework had its own tool abstraction and integrations were framework property. After it, a tool is a server anyone can talk to. This changed the economics of the whole category — see 2.074.

Structured output. Forcing the response into a declared shape and validating it.

Prompt optimization. Automatically searching for a better prompt against a metric you defined. A build-time activity, not a runtime one.

The four shapes of tool in this category#

1. Durable workflow runtimes — run your sequence and survive reality. The value is resumption, retries with state, and long human pauses.
2. Declarative pipelines — describe components and connections; the structure is inspectable before it runs.
3. Typed thin layers — leave control flow in ordinary code, validate the edges.
4. Compilers — improve the prompt at build time against a metric. Composes with any of the above.

And a fifth option that is not a tool: write it yourself. In 2026 that is a real answer for simple flows, because the tool boundary is now a protocol and the integration argument for frameworks has weakened.

How this changed recently, and why old advice misleads#

Two things happened that make writing from 2024 or 2025 actively misleading:

Everything pivoted to agents. Between late 2025 and mid 2026, essentially every major framework in this category re-centred on model-driven agents — deprecating older patterns, merging products, putting agents at the center of major releases. Advice that names a “best orchestration framework” from that era is often naming a product that has since changed shape or owner.

Tools became a protocol. MCP became the default way to expose tools. The main practical argument for adopting a big framework — “it has hundreds of integrations” — is much weaker when the integrations are servers your own code can call.

The net effect: the reasons to adopt a framework narrowed to control flow, and control flow narrowed mostly to durability.

The question to ask first#

Before comparing anything:

What happens when the process dies mid-run — and does anything wait for a human?

If the answer is “we retry the whole job, it’s fine”, most of this category’s complexity is not for you, and a small typed layer over the provider SDK will serve. If the answer makes you uneasy, you have found your requirement, and the field narrows to two options.

That question takes ten minutes and is worth more than any feature comparison.

The hardware-store analogy#

This survey is a hardware store, not a contractor. It describes what each tool is for and where it breaks, so you can pick for the job in front of you. It does not know your job.

Which is why the most useful thing here is not a ranking but a boundary: who decides the next step, your code or the model? Answer that, and half the shelves stop being relevant.

S1 Approach — LLM Orchestration Frameworks#

Pass: S1 Rapid Discovery Re-researched: 2026-08-17 (previous S1: 2025-11-19) Registry baseline: all versions verified against PyPI and npm on 2026-08-17

The boundary test#

Before anything can be compared, this category has to be separated from the one next door, because in 2026 they overlap enough that most published comparisons mix them freely — and a reader who arrives with a real decision gets a list containing three things that do not do the same job.

Who decides the next step — your code, or the model?

That single question sorts the field:

The test is about control flow, not about capability. A tool call does not make something an agent; a for loop does not make something a pipeline. The question is who holds the decision about what happens next, and the answer determines almost everything else about how you build, test, and debug the system.

Two consequences worth stating up front, because they shape the whole survey:

1. Most 2026 frameworks answer both, and sell themselves on the agent half. Every incumbent in this category shipped an agent story in the last twelve months. That does not make them the same product; it means each one must be read for what it offers the deterministic lane, which is often the older and quieter part of the documentation.
2. The strongest option in this category is sometimes not to adopt one. Direct API calls plus MCP for the tool edge is a real answer in 2026 in a way it was not in 2024, and it is profiled here as a peer rather than mentioned as a caveat. See no-framework.md.

Scope#

In scope — anything whose job is to run a sequence of model calls that you laid out:

• Workflow and pipeline runtimes (graphs, DAGs, suspend/resume, durable state)
• Prompt compilers and optimizers (systems that improve a prompt against a metric)
• Typed request/response layers over provider APIs (validation, retries, structured output)
• The null option: provider SDK plus MCP, no framework

Out of scope — covered by neighbouring surveys, and deliberately not duplicated:

Where a framework spans the boundary — and most do — this survey profiles only the orchestration half and points at the neighbour for the rest.

What changed since the previous pass#

The 2025-11 pass profiled five peers: LangChain, LlamaIndex, Haystack, Semantic Kernel, DSPy. Every one of those verdicts needed re-examination, and three needed replacing:

• LangChain reached 1.0 (2025-10-22) and deprecated AgentExecutor; LangGraph became the substrate rather than an add-on.
• Haystack reached 3.0 (2026-07-20) — a release explicitly about putting agents at the center, with a lighter core and components moved out to separate packages.
• Semantic Kernel was absorbed. Microsoft merged AutoGen and Semantic Kernel into the Microsoft Agent Framework. SK still ships, but recommending it as the Microsoft entry point — as the previous pass did — now points a reader at the wrong door.
• Three credible entrants appeared that the previous pass could not have seen: PydanticAI, Mastra (TypeScript), Agno.
• DSPy stopped being only a research tool. GEPA (ICLR 2026) turned prompt optimization into something that runs in CI against an eval suite.

Method#

1. Establish the boundary test above, and apply it to every candidate before profiling.
2. Verify every version and release date against PyPI or npm directly, on the day of writing. Vendor blogs and comparison articles are used for narrative and always marked as such; they are not used for facts that a registry can settle.
3. Profile each candidate in 100–200 lines, no code — S1 is for deciding whether something belongs on your shortlist, and code cannot help with that.
4. Mark every performance or adoption claim that comes from a vendor, a funded comparison, or a single blog post as unverified. This category is unusually noisy with benchmark marketing.
5. Produce a comparison matrix and a recommendation that answers by situation rather than by naming one winner.

A note on the evidence quality in this category#

More than in most surveys, the public material here is written by people with something to sell — framework vendors, consultancies with a preferred stack, and content farms producing “X vs Y in 2026” pages at volume. Several widely repeated figures in this space (framework speed multipliers, retrieval accuracy percentages, bug counts caught) trace back to a single unreplicated blog post. Where this survey repeats such a figure it is labeled; where a claim could be checked against a registry, an issue tracker, or a changelog, it was.

Comparison matrix#

All versions verified against PyPI or npm on 2026-08-17. Every entry here passes the boundary test in approach.md — your code decides the next step. Model-driven frameworks are in 1.201.

Release state#

Capability#

Fit#

The three questions that actually decide it#

Most of the matrix above is detail. In practice the choice falls out of three answers:

1. Must a run survive a restart, or pause for a human? Yes → LangGraph, or a durable workflow engine (Temporal/Inngest) with the null option. This is the single most discriminating question in the category, and only one framework here answers it natively.

2. What language is the surrounding application? TypeScript → Mastra, LangChain TS, or the Vercel AI SDK if streaming to a UI is the hard part. .NET or Java → Semantic Kernel / Microsoft Agent Framework. Python → everything.

3. Is the bottleneck plumbing, or output quality? Plumbing → one of the runtimes. Quality → DSPy, which composes with any of them and with none of them. These are different problems and the category routinely conflates them.

What the matrix does not measure#

Deliberately absent, because the available figures do not support the comparison:

• Latency overhead. The per-call overhead of every framework here is small relative to a model round-trip. Published millisecond figures (including those in the previous version of this survey) come from unspecified benchmarks and rank frameworks on a quantity that is not the bottleneck.
• Retrieval accuracy. Entirely corpus- and query-dependent; a single cross-framework percentage is not a meaningful number. See 1.204.
• Token efficiency. Depends on your prompts far more than on your framework.
• GitHub stars. Measures attention.

DSPy 3.x — a compiler, not an orchestrator#

Verified 2026-08-17. dspy 3.3.0, published 2026-08-03 (PyPI). dspy-ai is the same release under the older name. Originated at Stanford NLP.

What it is#

DSPy is the odd one in this category and the previous survey mis-shelved it. It is not a framework for wiring calls together — it is a compiler for prompts. You declare what a step should do in terms of inputs and outputs, supply a metric and a small set of examples, and DSPy searches for the prompt (and few-shot demonstrations) that maximise your metric. The artifact you deploy is an optimized prompt, not a runtime.

That distinction changes everything about when it is the right answer. Every other entry in this survey is a thing that runs at request time. DSPy’s work happens at build time; at request time you are running a string it produced.

What changed: GEPA#

The reason DSPy needs re-reading in 2026 is GEPA, presented at ICLR 2026 (accepted as an Oral) and shipping as dspy.GEPA. GEPA is a gradient-free optimizer that improves prompts by reflecting in natural language on what went wrong, rather than by policy gradients.

The reported results, from the paper: it outperforms reinforcement-learning approaches such as GRPO by up to 20% while using ~35× fewer model rollouts, and it optimises with 20–100 examples rather than thousands. It maintains a Pareto frontier across validation tasks, and is the default recommendation for complex DSPy workloads in 2026.

The practical consequence is the part worth internalising:

Optimization got cheap enough to run in CI, so the optimized prompt becomes a build artifact — versioned, diffable, and gated behind an eval suite like any other compiled output. The cost is reflection-LLM calls during compilation; inference cost is unchanged, because the output is a static prompt.

A team already running an eval suite can adopt DSPy as a build step without changing its serving path at all. That is a much lower-commitment adoption than “rewrite on a framework”, and it is why DSPy belongs on more shortlists in 2026 than in 2025.

Where it sits on the boundary test#

Neither side, and that is the point. DSPy does not decide the next step at request time; it decides what the prompt says before you ship. It composes with whatever runs your control flow — including nothing at all. A DSPy-optimized prompt called from a plain provider SDK is a perfectly coherent architecture, and arguably the most 2026-shaped one in this survey.

Strengths#

• Different axis of improvement. Everything else here reduces plumbing; DSPy improves output quality against a metric you chose. Those are not substitutes.
• Compiles to something portable. The artifact is a prompt, so you are not locked into a runtime.
• GEPA made it affordable — 20–100 examples and CI-scale cost, versus the thousands-of-rollouts regime of RL-based tuning.
• Composes with everything, including the null option.
• Academic provenance means the claims are published and reviewed, which is rare here.

Weaknesses#

• It requires a metric and examples. If you cannot say numerically what “better” means for your task, DSPy has nothing to optimize, and building that metric is usually the hardest part of adoption.
• Research-shaped documentation. The material assumes comfort with the vocabulary of optimization; a web developer wiring up a chatbot will find it steep.
• Smaller community than the incumbents, so fewer worked examples in any given domain.
• Compile-time cost is real even if inference cost is not — reflection calls against a strong model add up during iteration.
• The optimized prompt can be opaque: you get something that scores well, not something you would have written, which some teams find hard to review.

Choose it when#

• You already have an eval suite, or can build one
• Prompt quality is the bottleneck, not plumbing
• You want an artifact you can version and gate in CI
• The task is well-specified enough to score automatically (classification, extraction, structured judgement)

Look elsewhere when#

• You cannot define a metric — this is the disqualifying condition
• The problem is control flow, retries, or durability; DSPy addresses none of those
• You need a runtime → LangGraph, Haystack, or the null option
• Evaluation itself is the question → 1.205 LLM Evaluation

Previous-survey correction#

The 2025-11 pass listed DSPy alongside LangChain and Haystack as a peer framework and recommended it for “automated prompt optimization … research focus”, citing a 3.53 ms overhead figure. Treating it as a peer runtime was a category error — it is a build-time tool that composes with the others — and the overhead figure measures something that is not where DSPy’s cost lives. Both are corrected here.

Haystack 3.0 — the pipeline framework that moved to agents#

Verified 2026-08-17. haystack-ai 3.0.0, published 2026-07-20 (PyPI). Maintained by deepset.

What it is now#

Haystack was the production-minded pipeline framework of the previous generation: you declared components and the connections between them, and the pipeline ran. That description is now half true. Haystack 3.0, released 2026-07-20, is explicitly the release where agents move to the center of the framework.

The 3.0 changes fall into two groups.

Agents became the headline. The release added first-class skills, a general-purpose hooks system for controlling the agent loop, built-in run introspection, and prebuilt high-level agents for common jobs such as deep research. The Agent now owns tool execution end to end — the standalone ToolInvoker was removed.

The core got lighter. Legacy Generators are gone, haystack-experimental is no longer a core dependency, and roughly 30 components moved out into independently released integration packages. Two pipeline classes were merged into one.

That second group is the more interesting one for this survey. A framework that moves 30 components out of its core and merges its pipeline classes is a framework simplifying its deterministic layer, not abandoning it. The pipeline is still there, and it is smaller.

Migration#

deepset handled the 2.x → 3.0 transition better than this category usually manages:

• Breaking changes were kept deliberately small — a handful of legacy components removed, the lighter core, the merged pipeline classes. Most components are unaffected.
• 2.31 continues to receive security patches and critical bug fixes until end of October 2026, which is a real window rather than a courtesy.
• The migration guide ships with before-and-after examples and as an agent skill with a static scanner that flags v2 patterns — an unusually concrete piece of tooling, and a small sign of where the ecosystem is going.

If you are on 2.x today, you have a dated deadline and a scanner. That is more than most of this category offers.

Where it sits on the boundary test#

Historically deterministic, now both. Haystack’s pipeline remains a you-declare-the-graph model: components and connections, fixed at build time. That is this survey’s category, cleanly. The 3.0 agent additions are 1.201’s territory.

The practical reading: Haystack is one of the few frameworks here where the deterministic path is a first-class, well-documented product rather than the residue of an older design — but 3.0 signals that the vendor’s attention has moved, and a reader choosing it for pipelines should expect the agent surface to get the new features first.

Strengths#

• Production posture. deepset’s positioning has consistently been enterprise deployment rather than notebook demos, and it shows in the operational surface.
• The pipeline model is genuinely declarative — components and connections, inspectable as a structure, which makes a Haystack pipeline easier to reason about statically than an imperative chain.
• A real migration story, with a dated support window and a scanner.
• Lighter core in 3.0 reduces the dependency surface, which was a fair criticism of 2.x.
• On-premise and VPC deployment is a first-class concern, not an afterthought.

Weaknesses#

• The vendor’s attention has visibly shifted to agents. For a reader in the deterministic lane that is a trajectory risk, not a present-day defect.
• Smaller ecosystem than LangChain, and 3.0 moved components into separate packages, so integrations now come with their own release cadences to track.
• No durable execution story comparable to LangGraph’s. A Haystack pipeline is a run, not a resumable one; long-running or human-interrupted workflows need something else underneath.
• Community size is a fraction of LangChain’s, so unusual problems have fewer prior answers.
• The 2.x → 3.0 boundary means much of the searchable material is now for the previous major version.

Choose it when#

• The workload is a document-processing or retrieval pipeline with a shape you know
• Deployment is on-premise, in a VPC, or otherwise operationally constrained
• You want a declarative structure you can inspect rather than an imperative chain
• A dated migration window matters to your organization

Look elsewhere when#

• You need runs that survive a restart → LangGraph
• Retrieval strategy is the actual question → 1.204 RAG Pipelines
• You want the largest possible integration catalog → LangChain
• The model should be choosing the path → 1.201

Previous-survey correction#

The 2025-11 pass recommended Haystack for “Fortune 500 production deployment” on the strength of quoted latency and token figures (5.9 ms overhead, 1.57k tokens). Those figures came from a benchmark of the 2.x line with no published methodology, and the release they described is now a major version behind. The production positioning holds; the numbers are not reproduced here.

LangChain 1.x and LangGraph — the graph substrate#

Verified 2026-08-17. langchain 1.3.15 and langgraph 1.2.11, both published 2026-08-11 (PyPI). TypeScript langchain 1.5.9, published 2026-08-14 (npm).

What it is now#

LangChain and LangGraph reached 1.0 together on 2025-10-22, alongside a $125M Series B. The 1.0 line drew a division of labor that the project had been moving toward for two years: LangChain is the component library and the model/tool abstraction; LangGraph is the runtime that executes a graph you defined. In this survey’s terms, LangGraph is the orchestration engine and LangChain is what you build the nodes out of.

The 1.0 transition also retired the piece most associated with the old LangChain: AgentExecutor is deprecated and in maintenance until December 2026. New work is directed at create_react_agent() for the prebuilt path or, for anything custom, LangGraph’s StateGraph. If you have opinions about LangChain formed in 2023 or 2024, they are opinions about a codebase that no longer exists in that shape.

The claim that matters: durable state#

LangGraph 1.0’s organising idea is small enough to state in a sentence — an agent run should survive a server restart — and consequential enough to be the main reason to choose it. Execution state is checkpointed automatically. If the process dies mid-run, if a workflow waits three days for a human, or if a step needs to be retried after a transient provider failure, the run resumes from where it stopped rather than from the beginning.

That capability is what separates a workflow runtime from a for loop with retries, and it is the honest answer to “why not just call the API in a loop”. A loop loses everything when the process dies. For a long multi-step job, or anything with a human approval step in the middle, that difference decides the architecture.

Human-in-the-loop is a first-class API in the 1.0 line for the same reason: pausing for review, modification, or approval is just a durable interrupt, and once state is durable the pause costs nothing to hold.

LangGraph 1.2 shipped 2026-05-11; the current patch line (1.2.11) is the August 2026 release. The one notable deprecation is langgraph.prebuilt, whose functionality moved into langchain.agents.

Where it sits on the boundary test#

Both sides, and it is honest about which is which. A StateGraph where your edges are conditional on values your code computes is squarely this survey’s category. A prebuilt ReAct agent where the model picks the next tool is 1.201’s. The same library serves both, and the distinction shows up in your own code as whether the routing function reads a field you set or a decision the model made.

For a reader in the deterministic lane, this matters: LangGraph is usable as a plain durable workflow engine that happens to be good at LLM calls, and a large amount of its documentation and marketing is about the agent half you may not want.

Strengths#

• Durable execution is the real product, and nothing else in this category matches it without adding a separate workflow engine.
• Ecosystem breadth. The commonly cited figure is 700+ integrations; whatever the exact count, no competitor is close, and for an unusual provider or vector store the odds of an existing integration are best here.
• Graph-of-nodes has become the category’s shared vocabulary — Google’s ADK Go 2.0 independently converged on the same abstraction in 2026 — so the mental model transfers even if the library does not.
• The 1.0 release policy gives a stated deprecation path, which the pre-1.0 years conspicuously lacked.
• Both Python and TypeScript are maintained, with the TS line shipping on its own cadence.

Weaknesses#

• Two libraries, two mental models, one brand. Understanding what belongs in LangChain versus LangGraph is itself a learning cost, and the answer moved during the 1.0 transition.
• The abstraction tax is real when your use case is simple. If your workload is one prompt and one parse, the graph machinery is overhead you carry for nothing.
• Documentation churn. The 1.0 reorganisation invalidated a large body of tutorials, and search results still surface pre-1.0 patterns — including AgentExecutor examples that now lead new users into a deprecated API.
• Debugging a graph is harder than debugging a function. Durable state means state you must inspect somewhere other than a stack trace; in practice this pushes teams toward LangSmith, which is a commercial product from the same vendor.
• Venture funding cuts both ways: it buys maintenance and it creates pressure toward the hosted platform.

Choose it when#

• The run is long, resumable, or has a human approval step in the middle
• You need failure semantics stronger than “retry the whole thing”
• Breadth of integrations matters more than minimalism
• Your team is willing to learn a graph model and keep it

Look elsewhere when#

• Your workload is a handful of calls with straightforward control flow — the null option or a typed layer will be smaller and clearer
• The model, not your code, is meant to be choosing the next step → 1.201
• You want a small dependency surface; this is the largest in the category

Unverified claims seen in circulation#

• “3× faster prototyping” and similar multipliers, widely repeated from the previous survey generation onward, trace to vendor and blog comparisons with no published methodology. Not reproduced here.
• Star counts are cited constantly in this category and measure attention, not fitness.

LlamaIndex — the data-first framework, still pre-1.0#

Verified 2026-08-17. llama-index 0.14.23, published 2026-06-24 (PyPI). Note the version number and the date: it is the only major entry in this survey that has not reached 1.0, and the only one whose latest release is more than a month old.

What it is#

LlamaIndex has always answered a narrower question than LangChain: how do I get my data in front of a model? Ingestion, indexing, retrieval, and the query patterns built on them are the center of gravity, and the orchestration layer grew outward from that.

Its orchestration primitive is Workflows — an event-driven model where steps emit and consume events rather than being wired into a fixed graph. That is a genuinely different shape from LangGraph’s state machine: control flow emerges from what each step publishes, which is flexible and, for the same reason, harder to see whole.

Where it sits on the boundary test#

Both, with the deterministic side inherited from its indexing heritage. A query pipeline you declared is this survey’s category. Workflows can express either — an event-driven step that routes on a value your code computed is deterministic; one that routes on a model’s choice is not.

The more useful observation for a reader: if the reason you are looking at LlamaIndex is retrieval, the orchestration comparison is the wrong comparison. Read 1.204 RAG Pipelines and 1.206 Chunking Patterns first; the framework question is downstream of the retrieval question and much less consequential.

The version number is a real signal#

0.14.x after four years is not an accident and should be read carefully rather than dismissed. Two readings, both partly true:

• The generous reading: the project versions honestly, and declines to claim stability it does not have in a field moving this fast.
• The cautious reading: every competitor in this survey committed to a stability contract in the last year — LangChain 1.0, Haystack 3.0 with a dated support window, PydanticAI’s 1.0 API commitment — and this one has not. In a 0.x line, semantic versioning promises you nothing about the next minor release.

The release cadence supports the cautious reading being worth planning around: 2026-06-24 is comparatively stale in a category where four other packages shipped in the fortnight before this survey.

Strengths#

• Best-in-category for the data path. Connectors, index structures, and retrieval patterns are deeper here than anywhere else in this survey.
• Event-driven Workflows are a good fit for pipelines whose shape depends on what the data turns out to be — branchy ingestion, conditional enrichment.
• LlamaHub’s connector breadth is a genuine asset for pulling from many sources.
• Strong documentation for the RAG path specifically.

Weaknesses#

• Pre-1.0 with no published stability contract, alone among the major entries here.
• Slower release cadence than every peer as of this writing.
• Event-driven flow is harder to reason about statically than a declared graph — you cannot read the control flow off the structure, because there is no structure to read.
• Retrieval gravity. If your problem is not retrieval-shaped, you carry a large data-oriented surface for orchestration you could get more directly elsewhere.
• No durable-execution story comparable to LangGraph’s.

Choose it when#

• Retrieval over your own corpus is the core of the application
• Ingestion is branchy and data-dependent
• The connector catalog covers sources you would otherwise write by hand

Look elsewhere when#

• Orchestration is the actual problem and retrieval is incidental
• You need a stability contract to plan against
• Runs must be durable → LangGraph
• The retrieval strategy is the open question → 1.204

Previous-survey correction#

The 2025-11 pass recommended LlamaIndex on the strength of “35% better retrieval accuracy”. That figure has no traceable methodology, no stated corpus, and no baseline — retrieval accuracy is entirely dependent on corpus and query distribution, so a single percentage across frameworks is not a meaningful quantity. The qualitative claim, that LlamaIndex’s retrieval tooling is the deepest in this group, is retained; the number is withdrawn.

Mastra — the TypeScript answer#

Verified 2026-08-17. @mastra/core 1.59.0 and the mastra CLI 1.25.0, both published 2026-08-14 (npm). Built by the team behind Gatsby.

Why it is in this survey#

The previous pass profiled five Python packages and did not ask what a TypeScript team should do. That was a gap: a large share of the applications calling models are web applications, and telling their authors to stand up a Python service to run a chain is advice with a real cost attached.

Mastra is a TypeScript-native framework covering agents, workflows, tools, RAG, memory, evaluation and telemetry in one typed API surface. LangChain also ships a maintained TypeScript line (langchain 1.5.9, 2026-08-14) — the choice on that side of the fence is real, not a Python-or-nothing situation.

Where it sits on the boundary test#

Both, with a workflow story worth taking seriously. Mastra’s docs expose workflows with suspend/resume, snapshots, run watching, and tracing — the vocabulary of durable execution rather than of a chain. Its documented route to full durability is integration with a workflow engine (Inngest examples wrap Mastra agents), which is an honest design: it does not claim to be a durable execution engine itself, it composes with one.

Strengths#

• Native TypeScript, end to end. Types across agents, workflows and tools, in the language the surrounding application is already written in. No service boundary introduced purely to reach a Python library.
• One cohesive surface instead of six libraries with mismatched abstractions — workflows, memory, RAG, evals and OpenTelemetry traces in a single package.
• Suspend/resume and snapshots put it ahead of most of this survey on the deterministic-workflow axis, and it composes with a real durable engine for the rest.
• A local Studio environment for inspecting runs, which addresses the debuggability complaint that dogs graph frameworks.
• Very active release cadence (1.59.0 on 2026-08-14).

Weaknesses#

• Younger than the Python incumbents, with a correspondingly thinner body of production experience to learn from.
• Ecosystem is TypeScript-sized. The Python side of this category has years more accumulated integrations, and the ML tooling a data team expects is largely absent.
• Breadth is a risk as well as a feature. A single package covering agents, RAG, memory, evals and telemetry has more surface to keep current than a focused library, and each of those areas has specialists it must be measured against.
• Durability is delegated, so a team needing it adopts a second system anyway.
• The “default starting point for TypeScript AI applications in 2026” framing that recurs in write-ups is unverified positioning, not a measured adoption fact.

Choose it when#

• The application is TypeScript and adding a Python service would be a new operational burden
• You want workflows, memory and tracing without assembling four libraries
• Suspend/resume matters but full durable execution can be delegated
• The team values a typed surface and a local inspection environment

Look elsewhere when#

• The team is already Python — the depth is there, not here
• You need the largest integration catalog → LangChain
• Full durable execution is a hard requirement → LangGraph, or a workflow engine directly
• Retrieval is the actual problem → 1.204

The TypeScript sub-decision#

For a TS team the practical shortlist is Mastra, LangChain’s TypeScript line, or the null option with a validation library. The considerations mirror the Python side: breadth of integrations favors LangChain, cohesion and DX favor Mastra, and a small application with three model calls favors neither.

The null option — provider SDK, MCP, and no framework#

Profiled as a peer, not a caveat. In 2026 this is a defensible default for a large share of applications, and the reasons it became defensible are specific and datable.

What changed#

The case for adopting an orchestration framework in 2023–24 rested on three arguments. Two of them have expired.

“Frameworks give you provider portability.” Weakened. Provider SDKs converged on similar shapes, and the differences that remain — the ones that actually bite — are capability differences that no abstraction hides. A framework that lets you swap providers in one line still leaves you re-testing every prompt.

“Frameworks give you tool integrations.” Largely expired, and this is the big one. MCP has become the default tool protocol, with LangChain, LangGraph, LlamaIndex and CrewAI all moving from experimental support to treating it as standard. When the tool edge is a protocol, the integration catalog stops being a reason to adopt any particular framework — you point your own code at the same MCP server the framework would have used. The 2026-07-28 revision of the spec (see 2.074) added a stateless core and first-class extensions, which further reduces what a client library must provide.

“Frameworks give you control flow you would otherwise write.” Still true, and now the only durable argument. It is strongest where the control flow is genuinely hard: resumability, human interrupts, fan-out with join semantics, retries with state. It is weakest where the control flow is a sequence of three calls.

What the null option looks like#

Provider SDK for the model calls. MCP clients for tools that more than one thing will use. Ordinary functions, ordinary retries, ordinary logging. Where structured output matters, a validation library — which is why PydanticAI sits so close to this option that the boundary is mostly a matter of taste.

A related pattern gaining traction in 2026 is worth naming because it points the same direction: letting the model use CLI tools directly through a shell rather than wrapping them behind tool schemas, on the argument that models have seen enormous amounts of CLI documentation and already know these interfaces. Cloudflare’s “code mode” and Anthropic’s work on code execution with MCP are the visible versions. Whatever one makes of the specific technique, it is evidence for the same underlying shift: the wrapper layer is under pressure from both directions.

Strengths#

• Nothing to learn, nothing to migrate, nothing to deprecate under you. In a category where every incumbent shipped a major version in the last year, this is not a small benefit.
• Debuggable. A stack trace points at your code. No graph state to inspect, no framework internals between you and the provider.
• Smallest dependency surface, which matters more in regulated or security-reviewed environments than the framework discussion usually admits.
• You keep the whole API. Frameworks lag provider features; direct calls do not.
• Composes with the parts worth having — a DSPy-optimized prompt and an MCP tool edge both work fine without a runtime.

Weaknesses#

• You will write control flow yourself, and if the requirements grow you will write a worse version of LangGraph, six months later, with no documentation.
• No durability. Process dies, run dies. Adding real durability means adopting a workflow engine anyway — at which point the question becomes which one.
• No shared vocabulary. A new hire cannot look up how your pipeline works.
• Retrieval, memory, and evaluation are all assembled by hand.
• The failure mode is gradual: it works until it does not, and the migration happens under deadline.

Choose it when#

• The application makes a handful of calls with control flow you can hold in your head
• The team is small and the code is read more often than it is extended
• Dependency surface is under scrutiny
• You want to defer the framework decision until the requirements are actually known — which is a legitimate strategy, not indecision

Look elsewhere when#

• Runs must survive restarts, pause for humans, or fan out and join → LangGraph
• You are already writing a scheduler, a retry policy, and a state store → adopt one
• The team is large enough that shared vocabulary beats minimalism

The honest test#

Ask what the framework would do for you that you are not already doing. In 2026 the answers that survive scrutiny are durability, the graph vocabulary for a team, and breadth of exotic integrations. If none of those is your problem, the framework is overhead — and the previous generation of this survey, like most writing in the category, did not make that possibility visible enough.

Other contenders — checked, not shortlisted#

Each of these was examined and left off the main shortlist for a stated reason. A reader whose situation matches one of the reasons should promote it.

Agno#

agno 2.9.0, published 2026-08-13 (PyPI). Describes itself as “the programming language for agentic software”, and the description is a fair guide to where it sits: the emphasis is on getting multi-agent systems running quickly, and the write-ups that recommend it consistently recommend it for fast multi-agent prototypes.

By this survey’s boundary test that puts it on the other side of the line — the model is meant to be deciding — so it belongs to 1.201 LLM Agent Frameworks rather than here. Actively released, worth watching, out of scope.

Instructor#

The narrowest useful tool in the neighbourhood: it takes a provider SDK call and returns a validated structured object, with retries on invalid output. No orchestration at all.

It occupies almost exactly the position described in no-framework.md — direct calls plus validation — and for teams whose only complaint about the null option is unstructured output, it is a smaller step than adopting PydanticAI. Left off the shortlist because it is a component rather than an approach, and the approach is already profiled.

Vercel AI SDK#

The other serious TypeScript option, particularly for applications where the model output is streamed into a UI. Its center of gravity is the frontend integration — streaming, React bindings, the rendering path — rather than multi-step orchestration.

A TypeScript team choosing between it and Mastra is really choosing between a UI-streaming-first library and a workflow-first one. If the hard part is what the user sees while tokens arrive, that is the Vercel SDK’s home ground and Mastra is the wrong tool; if the hard part is what happens across six steps, the reverse.

Durable workflow engines: Temporal, Inngest, Prefect#

Not LLM frameworks at all, and that is precisely why they belong in this list.

If the only reason you are shopping for an orchestration framework is durability — runs that survive restarts, retries with state, long human-in-the-loop pauses — then a general-purpose durable execution engine solves that problem directly, has solved it for much longer than any LLM framework, and does not couple your control flow to a fast-moving AI library. Mastra’s documented durability route is exactly this: wrap it with Inngest.

The composition worth considering, and rarely presented as an option in this category’s comparison articles:

Durable engine for control flow + provider SDK for the calls + MCP for tools + optionally a DSPy-compiled prompt. No LLM orchestration framework anywhere in it.

That stack is more work to assemble and much more stable to own. It is the right answer more often than the framework discourse suggests, and it is why no-framework.md is profiled as a peer.

Explicitly out of scope#

Named here because readers arrive expecting them, with a pointer to where each is actually covered:

The previous pass of this survey listed LangSmith, Langfuse and Phoenix in its own keywords, which is a small symptom of the boundary problem this rewrite exists to fix: without a control-flow test, everything adjacent to an LLM call looks like it belongs in the same comparison.

PydanticAI — types instead of a framework#

Verified 2026-08-17. pydantic-ai 2.31.0, published 2026-08-15 (PyPI) — the most recently released package in this survey. From the Pydantic team; v1.0 shipped September 2025 with an API stability commitment.

What it is#

PydanticAI did not exist when the previous pass was written, and it represents the clearest alternative position in the category: the problem is not that model calls need orchestrating, it is that they are untyped.

The pitch its authors use is “the FastAPI feeling applied to GenAI”, and the analogy is load-bearing. FastAPI did not orchestrate your web application; it made request and response shapes into Python types and validated them at the boundary. PydanticAI does the same for model calls: you declare the output as a Pydantic model, and the library handles validation, retries on malformed output, and structured extraction — while your control flow stays ordinary Python.

The design goal, stated repeatedly by its maintainers, is that agent code should look like normal Python. There is no graph to declare and no DSL to learn. A for loop is a for loop.

Where it sits on the boundary test#

Squarely in this survey’s category, at the minimal end. Your code decides everything; PydanticAI types the edges. It also ships an agent surface — and calls itself an “AI Agent Framework” in its own summary — but the deterministic, typed-call use is the one that belongs here, and it is the one most teams adopt it for.

Strengths#

• Types catch a real class of failure. Structured output that does not match its schema is the most common production LLM bug, and it is exactly what a validation layer is for. Retry-on-invalid is built in rather than hand-rolled.
• Small surface, small commitment. Adopting it does not restructure your application, and removing it later means deleting a decorator, not unwinding a graph.
• API stability commitment since 1.0 — meaningful in a category where the incumbents spent years breaking imports between minor versions.
• The Pydantic team’s track record. Pydantic itself is a dependency of a large share of the Python data ecosystem; this is not a first project.
• The most actively released package here (2.31.0 on 2026-08-15), which cuts both ways but signals investment.

Weaknesses#

• Ecosystem breadth is not the point and not present. LangChain’s several-hundred integrations have no equivalent; you will write the glue yourself.
• No durable execution. If the process dies mid-run, the run is gone. For long or human-interrupted workflows this is disqualifying on its own.
• Version velocity. Reaching 2.31 within a year of 1.0 means a lot of change, and the stability commitment covers the API rather than the surrounding guidance.
• Doing less is the feature, but it means teams needing retrieval, memory, and observability will assemble those separately.
• Being newer, it has the thinnest body of production war stories in this survey.

Choose it when#

• The application is ordinary software that happens to call a model
• Structured, validated output is the requirement
• You want to keep control flow in plain Python and stay able to leave
• Your team already lives in Pydantic and FastAPI

Look elsewhere when#

• Runs must survive restarts or pause for humans → LangGraph
• You need breadth of prebuilt integrations → LangChain
• Prompt quality, not plumbing, is the bottleneck → DSPy
• The model should be choosing the next step → 1.201

Circulating claims, flagged as unverified#

The comparison content around PydanticAI is unusually promotional, and three figures recur without primary sources. They are recorded here because a reader will meet them, not because this survey endorses them:

• “MindsDB’s switch from LangChain to PydanticAI yielded a 10× performance improvement.” No published methodology; a framework swap alone rarely produces an order of magnitude, and such migrations usually coincide with rewrites.
• “8/10 vs LangChain’s 5/10 developer experience” in a “90-day benchmark” — a scored opinion from a single source.
• “The type system caught 23 bugs that would have reached production.” Uncounterfactual by construction.

The underlying qualitative claim — that validating output against a schema catches errors that would otherwise surface at runtime — is sound and does not need these numbers.

S1 Recommendation#

Re-researched 2026-08-17. This supersedes the 2025-11-19 recommendation entirely; the decision tree it contained pointed at three doors that have since moved.

Start here#

Do you need a framework at all?

Ask it first and answer it honestly, because in 2026 the reasons that used to make the answer automatic have thinned out. Provider portability is mostly illusory. Tool integration is now a protocol — MCP — that your own code can speak as well as any framework can. What is left, and what still justifies adopting one, is:

• durability — runs that survive a restart, retry with state, or pause for a human
• shared vocabulary — a team large enough that “it’s a graph” beats “read the module”
• breadth — an unusual provider or store where someone else already wrote the glue

If none of those is your problem, the null option plus a validation library is a real answer, not a compromise. See no-framework.md.

Then: three questions#

1. Must a run survive a restart, or pause for a human?

→ LangGraph (langgraph 1.2.11). Durable checkpointed state is its organising idea and nothing else here matches it natively. The alternative is a general-purpose durable engine — Temporal, Inngest — under the null option, which is more assembly and much more stable to own.

2. What language is the application written in?

• TypeScript → Mastra (@mastra/core 1.59.0) for workflows and cohesion; LangChain’s TS line for breadth; the Vercel AI SDK if streaming into a UI is the hard part. Do not stand up a Python service to reach a Python framework.
• .NET or Java → Microsoft Agent Framework for new work; Semantic Kernel remains supported and is the right answer for an existing deployment. Do not follow the previous version of this survey to Semantic Kernel as the Microsoft front door — that changed when Microsoft merged it with AutoGen.
• Python → everything below applies.

3. Is the bottleneck plumbing, or output quality?

• Plumbing → a runtime, chosen by the rest of this page.
• Quality → DSPy (dspy 3.3.0) with GEPA. It composes with any runtime and with none. The prerequisite is a metric: if you cannot score “better” automatically, DSPy has nothing to work with, and building that metric is the real adoption cost.

By situation#

What this survey will not tell you#

There is no single default. The previous pass named LangChain as the general-purpose answer, and that recommendation aged badly for a reason worth stating: the category fragmented along an axis it did not have in 2025. Durability, typing, compilation and breadth are now different products, and the framework that is best at one of them is mediocre at the others.

Anyone who tells you the answer without first asking whether your runs need to survive a restart is selling something.

Trajectory warning#

Every incumbent in this category shipped an agent story in the last twelve months — LangChain deprecated AgentExecutor in favor of graphs, Haystack 3.0 put agents at the center, Microsoft merged Semantic Kernel into an agent framework. The deterministic lane is not being abandoned, but it is no longer where the investment goes.

Choose accordingly: prefer the options whose deterministic story is a product — LangGraph’s durable graphs, Haystack’s declarative pipelines, PydanticAI’s typed calls — over ones where it is the residue of an earlier design. And see S4 for what this means over a two-year horizon.

Semantic Kernel — still shipping, no longer the front door#

Verified 2026-08-17. semantic-kernel 1.44.1, published 2026-08-06 (PyPI). Maintained by Microsoft. Also available for C# and Java.

The correction this profile exists to make#

The previous pass of this survey gave Semantic Kernel a clear recommendation: “Are you in the Microsoft ecosystem (Azure, .NET, M365)? → Use Semantic Kernel.” That advice now sends a reader to the wrong door.

Microsoft merged AutoGen and Semantic Kernel into the Microsoft Agent Framework, which reached v1.0 GA in April 2026 and is the recommended entry point for new work on that stack. (Version history and the merge are tracked in 1.201 LLM Agent Frameworks, which verified MAF 1.13.0 for Python on 2026-07-30.)

Two things are true at once, and both matter:

• Semantic Kernel is not dead. 1.44.1 shipped on 2026-08-06 — active, recent maintenance. It continues as the SDK layer, and existing deployments are not stranded.
• Semantic Kernel is no longer where Microsoft points new projects. Recommending it as the Azure/.NET answer in 2026 means recommending a layer rather than the product built on top of it.

Where it sits on the boundary test#

Both, historically deterministic. Semantic Kernel’s original model — functions, plugins, and a planner — put your code in charge of composition, which is this survey’s category. The planner and the agent work that followed moved toward the model deciding, which is where the AutoGen merge took it.

For a reader who wants the deterministic lane on Microsoft’s stack, SK remains a reasonable and supported choice. For anyone starting fresh and expecting to end up with agents, MAF is the door.

Strengths#

• Genuine multi-language parity — C#, Python, and Java. Nothing else in this survey offers a first-class .NET story, and for a Microsoft-shop team that single fact often decides the question.
• Azure integration is deeper than any third-party framework can offer.
• Enterprise support expectations come with the vendor, which matters to procurement in ways engineers routinely underestimate.
• Still actively maintained, with a recent release.

Weaknesses#

• Strategically superseded. New investment is going into Microsoft Agent Framework; SK’s role is now foundational rather than headline.
• Documentation and examples are split across SK, AutoGen’s legacy material, and MAF, and search results do not distinguish them reliably.
• Smaller Python community than the Python-native options; the center of gravity is .NET.
• Cloud gravity. The value concentrates on Azure, which is a strength in a Microsoft shop and a cost elsewhere.
• No durable-execution story comparable to LangGraph’s.

Choose it when#

• The organization is a Microsoft shop and .NET or Java parity is required
• Azure is the deployment target and its integrations are worth the coupling
• An existing Semantic Kernel deployment works and stability matters more than novelty

Look elsewhere when#

• Starting fresh on Microsoft’s stack and expecting agents → Microsoft Agent Framework, covered in 1.201
• Python is the primary language and Azure is not mandated → most of this survey
• Durability is the requirement → LangGraph

What to watch#

Whether Semantic Kernel’s maintenance continues at this cadence once MAF matures. A merged product usually means the underlying layer stabilises rather than grows; that is fine for a running system and a poor bet for a new one with a long horizon.

S2 Approach — Comprehensive Analysis#

Pass: S2 Comprehensive Re-researched: 2026-08-17

What this pass is for#

S1 decided which options belong on a shortlist. S2 asks how they actually work, so that a reader can predict how each will behave when the requirements change — which they will.

Four questions, one file each:

1. control-flow-models.md — what shape does each library impose on your program? This is the difference that survives every version bump.
2. durability-and-state.md — what happens when the process dies? The most discriminating question in the category, and the least discussed.
3. tool-edge-and-mcp.md — what does a framework still add now that MCP standardized the tool boundary? This is where the 2024 case for frameworks quietly expired.
4. production-readiness.md — versioning, migration, and what it costs to be wrong.

A deliberate constraint: minimal code#

This pass contains very little code, and that is a correction rather than an omission.

The previous version of this survey published roughly 18,000 lines, a large fraction of it verbatim API snippets. That had two costs. For a reader, the comparison drowned: you cannot see the difference between two architectures through four hundred lines of imports. For the published page, it meant search engines matched it on exact API symbols — the survey ranked in the top five for queries like "add_conditional_links" "querypipeline" and nowhere for “llm orchestration frameworks”, which is the question it exists to answer.

Code appears here only where the shape of an idea cannot be shown otherwise, and then in five to ten lines. Anything longer belongs in the library’s own documentation, which is maintained and versioned and this survey is not.

The methodology’s guidance — S2 is architecture plus minimal code, 300–500 lines — was correct and was not followed. It is followed here.

What is not compared#

• Millisecond overhead. Framework overhead is small against a model round-trip. The figures circulating in this category (including in the previous version of this file) come from unpublished benchmarks and rank options on a quantity that is not the bottleneck.
• Token efficiency. A function of your prompts, not your framework.
• Retrieval accuracy. Corpus-dependent; see 1.204.

Where a number would help and no trustworthy one exists, this pass says so rather than repeating one.

Control-flow models#

Four shapes exist in this category. Every library here is one of them, the shape outlives version numbers, and picking the wrong one is the mistake that costs a rewrite.

1. The state graph — LangGraph#

You declare nodes and the edges between them; a runtime walks the graph, carrying a state object. Routing is a function of state.

graph.add_node("fetch", fetch)
graph.add_node("summarise", summarise)
graph.add_conditional_edges("fetch", lambda s: "summarise" if s["docs"] else END)

What it buys: the control flow is data. Because the runtime holds the graph and the state rather than your call stack, it can checkpoint between nodes, resume after a crash, pause for a human, and replay. Nothing else in this survey can offer those, and they all follow from this one structural choice.

What it costs: your program’s logic no longer lives in the language’s own control flow. if becomes an edge, a loop becomes a cycle with a termination condition, and a stack trace tells you which node failed rather than how you got there. Teams consistently report the graph as the steepest part of adoption.

2. The declarative pipeline — Haystack#

You declare components and connect their outputs to inputs. The result is a structure that can be inspected — and, unlike a graph with conditional edges, largely a static one.

pipe.add_component("retriever", retriever)
pipe.add_component("prompt", builder)
pipe.connect("retriever.documents", "prompt.documents")

What it buys: you can read the whole flow off the declaration, draw it, and validate that the wiring type-checks before anything runs. For a pipeline whose shape is known in advance — ingest, embed, retrieve, generate — this is the clearest model in the category.

What it costs: shapes that genuinely vary at runtime fight the model, and the pipeline is a run rather than a resumable process.

3. Event-driven steps — LlamaIndex Workflows#

Steps declare which events they consume and emit; the flow is whatever the events imply.

What it buys: branchy, data-dependent processing where the path depends on what a document turns out to be. Adding a step means adding a consumer, not rewiring a graph.

What it costs: there is no structure to read. Understanding the flow means tracing which step emits what, across files. This is the classic trade of event systems and it applies unchanged.

4. Plain code with typed edges — PydanticAI, the null option, Mastra’s simple path#

Control flow is the language’s. The library types the boundary and validates what comes back.

result = await agent.run("extract the invoice fields", output_type=Invoice)

What it buys: every debugging tool you already own. A stack trace is a stack trace, a breakpoint works, a new engineer reads it top to bottom. The failure mode that a schema catches — a model returning something that does not fit — is caught at the boundary with a retry.

What it costs: everything the runtime would have done for you, you do. Retries with state, resumption, fan-out with a join: all yours to write, and the second time you write one you are building a worse LangGraph.

The odd one out: compile-time — DSPy#

DSPy has no request-time control-flow model, because it does not run your program. It takes a declared input/output signature, a metric and examples, and searches for a prompt. The artifact is a string; the runtime is whichever of the four above you chose.

This is why DSPy is not an alternative to the others but a layer that composes with any of them — including the null option.

How to choose the shape#

The shape is the decision. The library is downstream of it, and the version numbers in this survey will be stale long before the shapes are.

Durability and state — the question that decides the category#

If you read one file in this survey, read this one. What happens when the process dies? is the most discriminating question in the category and the least discussed in its comparison articles.

Why it discriminates#

Every framework here can call a model, parse a response, and call another. Where they diverge is what exists after a crash.

• In a plain loop, or a Haystack pipeline, or a LlamaIndex workflow: nothing. The run is in-process state. A restart loses it, and “resume” means “start again”.
• In LangGraph: state is checkpointed between nodes. The run resumes where it stopped.

That is not a feature difference, it is an architectural one. It follows from the state graph model — because the runtime holds the graph and the state instead of your call stack, it has something it can persist.

What durability actually gets you#

Four capabilities, all the same capability underneath:

Crash resumption. A deploy, an OOM kill, or a spot instance reclaim mid-run costs you the current node rather than the job.

Human-in-the-loop. A run that waits for approval waits for hours or days. Holding that in memory is not an option; holding it as a checkpoint is free. LangGraph 1.0 made human-in-the-loop first-class for exactly this reason — once state is durable, pausing is just not resuming yet.

Retry with state. A transient provider failure at step 9 retries step 9. Without durability, retrying safely means either replaying steps 1–8 (paying for them again, and hoping they were idempotent) or hand-rolling a state store.

Auditability. The checkpoints are a record of what the system did, which regulated environments tend to require and which is otherwise a logging project.

What it costs#

Not free, and the cost is under-discussed:

• A state store to operate. Checkpoints live somewhere — Postgres, Redis, a managed service. That is infrastructure with backup, retention and access-control questions attached.
• Serialisable state. What crosses a node boundary must be persistable. Open connections, file handles and closures cannot go in the state object, and discovering this late forces refactors.
• Debuggability moves. The truth about a run is in the checkpoint store, not the stack trace, which pushes teams toward a tracing product — in LangGraph’s case, LangSmith, from the same vendor.
• State versioning. A checkpoint written by yesterday’s code must be readable by today’s, or in-flight runs break on deploy. This is a real migration problem and it arrives the first time you change the state shape.

The alternative: a general-purpose durable engine#

Durability is not an LLM problem. Temporal, Inngest and Prefect have solved it for years, for workloads far more demanding than a chain of model calls, and they do not couple your control flow to a library that ships a major version every nine months.

The composition:

Durable engine for control flow + provider SDK for the calls + MCP for tools.

Mastra’s own documented route to durability is exactly this — its workflows offer suspend/resume and snapshots, and full durable execution comes from wrapping with Inngest.

When the engine wins: durability is your only reason to shop; you already run one; your workflow includes non-LLM steps that must be equally durable; you want the AI dependency kept shallow.

When LangGraph wins: you want durability and the LLM-shaped ecosystem in one thing; the team benefits from a shared graph vocabulary; the integration catalog matters.

The honest test#

Ask two questions in order:

1. Does a lost run cost real money or real trust? A dropped chat completion is a retry. A dropped six-step document workflow that already made three paid API calls and sent an email is not.
2. Does anything wait for a human? If yes, you need durability, and no amount of careful try/except substitutes for it.

If both answers are no, the frameworks that lack durability are not deficient — they are simply not solving a problem you have, and their smaller surface is a genuine benefit.

If either answer is yes, this single requirement narrows the category to LangGraph or a durable engine, and most of the rest of the comparison stops mattering.

Production readiness — versioning, migration, and the cost of being wrong#

The question this file answers is not “does it work” — all of them work — but what does it cost you when the library changes? In a category where every major package shipped a breaking release in the last twelve months, that is the operational risk that matters.

Stability contracts, ranked#

Haystack’s approach deserves the top slot for a specific reason: a dated support window plus an automated scanner converts “we broke things” into a schedulable task. That is what a migration story should look like, and it is rarer in this category than the maturity of these projects would suggest.

The migration debt nobody prices#

Adopting a framework means signing up for its upgrade treadmill. Two recent examples show the shape:

LangChain 1.0 deprecated AgentExecutor — the single most-copied pattern from three years of tutorials. The code still runs until December 2026. The tutorials, blog posts and Stack Overflow answers that teach it will not be corrected, so search results actively teach new engineers a deprecated API. The migration cost is small; the documentation confusion cost is large and lands on whoever onboards next.

Haystack 3.0 removed ToolInvoker, merged two pipeline classes and moved ~30 components into separately released packages. The last of those is the sleeper: those components now have their own version numbers and release cadences to track. A lighter core is good engineering and it converts one dependency into several.

What to check before adopting#

1. Is there a dated support window for the version you are on? Haystack, yes. LangChain for the deprecated path, yes. Others, generally no.
2. How much of the searchable material describes the current major version? For LangChain and Haystack, a minority of it does. Budget for engineers finding stale answers.
3. What is the exit? Ask concretely: if this library were abandoned tomorrow, what would you rewrite? For the null option and PydanticAI, very little. For a state graph with checkpoints in the framework’s format, a great deal.
4. Who else depends on it? A framework in your dependency tree constrains your provider SDK versions too, and conflicts here are common.

Release cadence, observed 2026-08-17#

Recency is not quality, but a package that has not shipped in two months in this field is saying something:

Read this alongside the stability column, not instead of it. Very high cadence and a strict stability contract is the ideal combination (PydanticAI); very high cadence without one is churn you absorb.

The recommendation this file implies#

Prefer options whose blast radius you can state. If you cannot answer “what would we rewrite if this went away”, the coupling is deeper than you think — and in this category, where the whole field re-pointed at agents within a year, that answer is worth having before you need it.

S2 Recommendation#

S1 asked which options belong on a shortlist. S2 looked at how they work. Three findings change the decision, and none of them appears in the comparison articles this category generates.

1. Choose the control-flow shape, not the library#

Four shapes exist — state graph, declarative pipeline, event-driven steps, plain typed code — and the shape is what you actually commit to. Version numbers in this survey will be stale within months; the shapes will not.

Practical consequence: if you find yourself comparing feature checkboxes between two libraries of the same shape, the decision is low-stakes and you should pick on ecosystem and stability. If you are comparing across shapes, that is the real decision and deserves the time.

2. Durability is the discriminator; everything else is negotiable#

One question separates this category more cleanly than any feature matrix:

Does a lost run cost real money or real trust, or does anything wait for a human?

• Yes → LangGraph, or a general-purpose durable engine (Temporal, Inngest) under the null option. Nothing else here answers it natively, and no amount of careful exception handling substitutes.
• No → the frameworks without durability are not deficient; they are not solving a problem you have, and their smaller surface is a real benefit.

Everything else — integrations, typing, developer experience — is a preference. This one is an architecture.

3. MCP removed the main reason to adopt a framework#

The tool catalog was the 2024 case for frameworks, and it has largely expired: MCP is now the default tool protocol across the incumbents, and your own code can speak it. What a framework still adds at that boundary is lifecycle management, schema-to-signature glue, and result handling in the flow — real but much smaller than a catalog count implies.

Practical consequence: discount integration counts heavily, and adopt MCP at the boundary regardless of what you choose above it. The protocol is the portable part; the framework is not.

Revised decision path#

Does a run need to survive a restart, or pause for a human?
├─ YES → LangGraph, or durable engine (Temporal/Inngest) + provider SDK
└─ NO
   ├─ Is the pipeline shape fixed and known?      → Haystack 3.0
   ├─ Is the path data-dependent and branchy?     → LlamaIndex Workflows (read 1.204 first)
   ├─ Is the app TypeScript?                      → Mastra / LangChain TS
   ├─ Is it ordinary code needing typed output?   → PydanticAI
   └─ Is the flow simple enough to hold in mind?  → no framework + validation library

Separately, on any of the above:
   Is prompt quality the bottleneck and can you score it?  → DSPy 3.3 + GEPA

What to do first#

Write down what happens when the process dies. If the answer is “we retry the whole job and that is fine”, you have just eliminated most of this category’s complexity and can choose on ergonomics. If the answer makes you uncomfortable, you have found your requirement, and the shortlist is two items long.

That question costs ten minutes and is worth more than any feature comparison in this survey.

Where the risk sits#

Not in picking the “wrong” library — most are recoverable — but in three specific places:

1. Coupling your control flow to a framework’s format. Checkpoints in a proprietary shape are the deepest coupling here. Know the exit before you take it.
2. Choosing on an integration count that MCP has already made cheap.
3. Choosing a framework whose deterministic story is residue. See S4: every incumbent re-pointed at agents in the last year, and the deterministic lane is maintained but no longer where the investment goes.

The tool edge after MCP#

The strongest argument for adopting a framework in 2024 was the integration catalog. That argument has largely expired, and understanding why is the key to reading this category in 2026.

What happened#

MCP became the default tool protocol. LangChain, LangGraph, LlamaIndex and CrewAI all moved from experimental MCP support to treating it as the standard way tools are reached. The 2026-07-28 revision of the specification — the largest since launch — added a stateless core, per-request capability negotiation, and first-class extensions (Tasks for long-running work, MCP Apps for inline UI). Full analysis in 2.074 MCP Protocol.

The consequence for this survey is structural:

When the tool boundary is a protocol, the tool catalog stops being a property of a framework. Your own code can speak MCP as well as any framework can.

A framework’s 700 integrations are worth much less when the ten you need are MCP servers that any client can call. This is the single largest change in the case for adopting an orchestration framework since the previous version of this survey, and it moves in one direction: against.

What a framework still adds at the tool edge#

Not nothing. Being precise about the remainder is the useful exercise:

Lifecycle management. Connecting to several MCP servers, handling reconnection, negotiating capabilities per request, and surfacing failures coherently is a modest but real amount of code that a mature client handles for you.

Schema-to-signature glue. Turning a tool’s declared schema into something your program can call with type checking — PydanticAI’s core competence, and something the raw protocol leaves to you.

Result handling in the flow. Deciding what a tool result does — becomes state, triggers a branch, gets retried — is control flow, and that is the framework’s actual job.

Non-MCP integrations. Vector stores, document loaders, provider quirks. These are not tools in the MCP sense and remain genuine catalog value, especially LlamaIndex’s connectors.

The counter-current: code mode#

A visible 2026 pattern pushes further in the same direction. Rather than wrapping tools in schemas at all, let the model use CLI tools through a shell — the argument being that models have absorbed enormous amounts of CLI documentation and already know these interfaces without being told. Cloudflare’s “code mode” and Anthropic’s work on code execution with MCP are the prominent versions.

Whether or not the specific technique holds up, it is evidence for the same shift: the wrapper layer is under pressure from both sides. Below it, a protocol standardized the tool boundary. Above it, models got good enough to use interfaces that were never designed for them.

What this means for a decision#

• Do not choose a framework for its integration count. Check whether the specific things you need are MCP servers. If they are, that column of the comparison is close to irrelevant.
• Do choose a framework for control flow — what happens with the result, and what survives a crash. That is the part MCP does not touch.
• Adopt MCP at the boundary regardless of framework choice. It is the portable part of the stack. Frameworks will keep changing shape; a tool server behind a stable protocol outlives them.

In practice, nobody picks one pattern#

A single request commonly shells out to a CLI, calls two MCP servers, runs a validation function, and hits a direct API for a status check. The patterns compose, and a framework that insists everything be wrapped in its own tool abstraction is adding friction rather than removing it.

This is the practical reading of the whole category in 2026: the tool edge is solved and standardized; the control flow is what you are actually choosing.

S3 Approach — Need-Driven Discovery#

Pass: S3 Need-Driven Re-researched: 2026-08-17

Method#

S1 and S2 judged the category on its merits. S3 asks a different question: given who you are and why you are here, what changes?

Six personas follow. Each states WHO — the team, its constraints, the shape of its codebase — and WHY it is looking, because the reason someone is shopping usually determines the answer more than any feature does. A team that arrives because a run keeps dying at step nine needs something categorically different from a team that arrives because a prompt keeps returning malformed JSON, even though both will read the same comparison articles.

The personas are written to be recognisable rather than exhaustive. If none fits, the question in recommendation.md — what happens when the process dies? — sorts most cases on its own.

A note on neutrality#

These personas describe categories of reader, not any particular project. Where a persona resembles a real situation, it is because the situation is common: web teams adding an AI feature, back offices automating document work, and TypeScript product teams are the three largest populations calling models in 2026, and none of the comparison material in this category is written for the first or third.

What the personas are testing#

Each one is a test of a different claim from S2:

Persona: the document back office#

WHO#

A team automating work that used to be done by people reading documents — invoices, claims, contracts, permit applications. Ten to forty steps end to end, some deterministic (fetch, validate, write to the system of record) and some model-driven (extract, classify, summarize). A human signs off before anything irreversible happens.

WHY they are here#

Not curiosity. Runs are dying partway through, and each failure costs money: paid API calls already made, a partially updated record, and someone re-keying the job by hand. Nobody can answer “where did run 4471 stop” without reading logs.

What actually matters to them#

• Resumption. A run that dies at step 9 must resume at step 9, not step 1.
• The human pause is not optional. Approval can take a day. Holding it in process memory is not a design.
• Auditability. Regulated or not, someone will eventually ask what the system did to a specific document on a specific date.
• Idempotency. Steps that spend money or send email must not run twice on a retry.

Recommendation#

LangGraph, or a general-purpose durable engine (Temporal, Inngest) with the provider SDK. This is the persona the durability discussion in S2 exists for, and it is the one case where the category’s answer is short.

Choose between them on a single question: is the rest of the workflow LLM-shaped?

• Mostly model calls, with the LLM ecosystem worth having in one place → LangGraph
• A long business process with a few model calls in it → durable engine, and keep the AI dependency shallow

Why not the others#

Haystack, LlamaIndex, PydanticAI and the null option all lose the run when the process dies. For this team that is disqualifying, not a trade-off. No amount of careful try/except reconstructs state that was never persisted.

What they must plan for#

• A state store to operate — Postgres or equivalent, with backups and retention. This is real infrastructure, and pretending otherwise is how the durability decision goes wrong.
• Serialisable state. Open file handles and connections cannot cross a node boundary. Discovering this late forces a refactor.
• Checkpoint versioning. In-flight runs must survive a deploy that changed the state shape. This problem arrives the first time and is easy to miss in testing.

The trap#

Building this on a framework without durability and adding “resume” later. Resumption is not a feature you bolt on — it is a property of holding state outside the call stack, and retrofitting it means rewriting the control flow. Decide it first.

Persona: the team whose problem is quality, not plumbing#

WHO#

A team that already shipped. The pipeline works, the plumbing is boring, and there is a test set — perhaps a few hundred labeled examples, perhaps a scoring rubric run by a model. Someone owns the accuracy number and reports it.

WHY they are here#

The number is not good enough. Classification accuracy sits at 82% and the business needs 90%. The team has spent six weeks hand-tuning prompts, gains are inconsistent, and each change risks regressing a case that used to pass. They are searching for a framework because that is what one searches for, but the framework is not the problem.

What actually matters to them#

• Measurable improvement, not a refactor.
• Not regressing the cases that already work.
• Reproducibility. A prompt improved by intuition on Tuesday cannot be defended on Friday.

Recommendation#

DSPy 3.3 with GEPA — and nothing else in this survey addresses their problem at all.

DSPy is a compiler, not a runtime. You declare a step’s inputs and outputs, supply the metric and examples you already have, and it searches for the prompt that maximises the metric. Their existing runtime — framework or plain code — does not change.

GEPA is why this is newly practical. Reflecting in natural language rather than using policy gradients, it reports beating RL approaches such as GRPO by up to 20% while using about 35× fewer rollouts, and it optimises with 20–100 examples rather than thousands. That moves optimization from a research project into something affordable in CI.

The shape to adopt:

Treat the optimized prompt as a build artifact: versioned in the repo, regenerated in CI, gated behind the eval suite. Compilation costs reflection-LLM calls; inference cost is unchanged, because what ships is a static prompt.

The prerequisite, stated plainly#

You must be able to score “better” automatically. If accuracy is a matter of taste, or the test set is twelve examples someone assembled once, DSPy has nothing to optimize. For most teams in this persona the honest first step is not adopting DSPy — it is building the eval set, which is also the harder half. See 1.205 LLM Evaluation.

What they should not do#

• Adopt an orchestration framework. Their plumbing works. Changing it will consume the quarter and move the accuracy number by nothing.
• Keep hand-tuning. Six weeks of manual prompt iteration without a metric is how teams arrive here; more of it is not the exit.
• Reach for fine-tuning first. Prompt optimization is cheaper, faster to iterate, and leaves the model swappable. Fine-tuning is a later and heavier answer — 1.208.

Why this persona is easy to miss#

Everything published in this category is about wiring. A team whose wiring is fine and whose outputs are mediocre reads a dozen framework comparisons and finds nothing that speaks to them, because the tool they need is filed under the same heading and does a completely different job. That mis-shelving was in the previous version of this survey too, which listed DSPy as a peer runtime and recommended it for “research”.

Persona: the regulated Microsoft shop#

WHO#

A team inside a bank, insurer, hospital system or government agency. C# is the primary language, some Java, Python only for data work. Azure is the deployment target, decided above this team’s level. Procurement requires vendor support agreements. Security review has opinions about every new dependency.

WHY they are here#

A business unit wants document summarisation or intake triage. The team must choose something that will pass architecture review, security review, and procurement — three gates that most of this category is not built to clear.

What actually matters to them#

• Language parity. The service will be maintained by C# engineers. A Python-only library means either a polyglot service or a team that cannot maintain its own code.
• A vendor to call. “It’s open source, file an issue” does not satisfy procurement.
• Dependency surface. Every transitive dependency is a review item and a CVE watch.
• Data residency. Where the calls go, and where state is stored, are compliance questions before they are engineering ones.

Recommendation#

Microsoft Agent Framework for new work; Semantic Kernel where a deployment already exists and works.

This corrects the previous version of this survey, which sent Microsoft shops to Semantic Kernel as the front door. Microsoft merged AutoGen and Semantic Kernel into MAF, GA since April 2026, and that is where new investment goes. SK is not dead — semantic-kernel 1.44.1 shipped 2026-08-06 — but it is now the layer beneath, not the destination. Full coverage of MAF is in 1.201.

Semantic Kernel remains the correct answer when: the deployment exists, it works, and the requirement is stability rather than new capability. Migrating a working compliant system to chase a merge is its own risk.

Why not the ecosystem leaders#

• LangGraph / LangChain: Python and TypeScript only. For a C# shop that is the end of the conversation regardless of ecosystem size.
• Haystack, LlamaIndex, PydanticAI, DSPy: Python only.
• The null option deserves more consideration here than anywhere else in this survey. A .NET team calling the Azure OpenAI SDK directly, with MCP at the tool edge, has the smallest dependency surface available and the easiest security review. If the control flow is simple, this is the strongest answer, not the fallback.

What to raise in architecture review#

• Where checkpoints live, if durability is adopted. State persisted by a framework is data subject to the same residency and retention rules as anything else.
• Which model endpoints are reachable and whether the framework can be constrained to approved ones.
• The exit. If the library were abandoned, what is rewritten? For a direct SDK plus MCP, very little — an answer that reviewers like and that is true.

The realistic path#

Start with the direct SDK and MCP. Adopt MAF when the control flow genuinely outgrows plain code — which, for intake triage and summarisation, it often does not. The pressure in a regulated environment is to choose the enterprise-branded framework early; the smaller surface is usually the easier system to defend.

Persona: the solo founder#

WHO#

One person, occasionally two. Building a product that has not found its shape yet. Every hour spent on infrastructure is an hour not spent finding out whether anyone wants the thing. No on-call, no review board, no legacy — and no colleague to inherit a decision that turns out badly.

WHY they are here#

They want to know what to build on so they do not have to rebuild in three months. The question behind the question is optionality: which choice keeps the most doors open?

What actually matters to them#

• Speed to a working demo, measured in days.
• Not being locked in before the product’s requirements are known.
• Not maintaining infrastructure that exists for a user base that does not.
• Cost. Every framework here is free; the model calls are not, and compile-time optimization loops can surprise you.

Recommendation#

Start with no framework. Provider SDK, a validation library, plain functions. Ship it.

For Python, PydanticAI is the one step up worth taking early: it is small, it types the boundary, and if it were deleted tomorrow the rewrite is a few functions. For TypeScript, the Vercel AI SDK if the product is a streaming chat UI, Mastra if multi-step flow shows up early.

Why deferring is a strategy, not procrastination#

The requirements that justify a framework — durability, shared vocabulary, exotic integrations — are all requirements you do not have yet and cannot predict. A solo founder adopting LangGraph in week one is buying insurance against a specific future, priced in learning time, at the moment when learning time is the scarcest input they have.

The counter-argument — “you will have to migrate later” — is weaker than it sounds. The migration is from your own small pile of functions, which you understand completely, to a framework whose requirements you will by then know precisely. That is a much easier migration than the reverse, and it is the one direction of this decision that does not involve unwinding somebody else’s abstractions.

The one exception#

If the product’s core loop is inherently long-running with human checkpoints — something like an agent that drafts work and waits for approval — then durability is a requirement from day one, not an emergent one. In that case take LangGraph or a durable engine at the start, because retrofitting resumption means rewriting the control flow. See the document back office persona.

Watch for#

• Framework-shaped procrastination. Evaluating five frameworks is more comfortable than finding out whether the product works. The evaluation feels like progress.
• Building on a demo. Tutorial code becomes the architecture faster than anyone intends.
• Adopting for the résumé. Real, and expensive to the product.

Persona: the TypeScript product team#

WHO#

A product team shipping a Next.js or SvelteKit application. Everything is TypeScript — frontend, API routes, background jobs. They deploy to Vercel, Fly, or a container platform. There is no Python anywhere in the stack and no appetite for adding it.

WHY they are here#

The product needs a multi-step AI feature: retrieve, then generate, then check, then stream the result to the user. They have read that “the AI ecosystem is Python” and want to know whether they must stand up a Python service — an entire second runtime, deploy target, and on-call surface — to use a decent framework.

What actually matters to them#

• Not adding a service. A Python sidecar is a permanent operational tax paid for one feature.
• Types across the boundary. The rest of the codebase is typed; an untyped island is a bug factory.
• Streaming into the UI. What the user sees while tokens arrive is a product decision, not an afterthought.

Recommendation#

No Python service is needed. The TypeScript side of this category is real:

• Mastra (@mastra/core 1.59.0) — workflows with suspend/resume and snapshots, memory, RAG and telemetry in one typed package, plus a local Studio for inspecting runs. The best fit when the hard part is multi-step flow.
• LangChain TS (langchain 1.5.9) — when integration breadth matters more than cohesion.
• Vercel AI SDK — when the hard part is streaming into the UI rather than orchestration. Do not use Mastra for what this does better.

Pick by which half is hard. If the difficulty is what the user sees while tokens arrive, that is the Vercel SDK’s home ground. If it is what happens across six steps, Mastra.

Where TypeScript genuinely lags#

Honest limits, so nobody is surprised later:

• Fewer integrations than the Python side, accumulated over fewer years.
• No DSPy equivalent. If prompt optimization against a metric becomes the bottleneck, that capability lives in Python, and this is the one case where a small offline Python step may earn its keep — at build time, not in the request path.
• Thinner production folklore. Fewer people have hit the sharp edges before you.

On durability#

Mastra offers suspend/resume and snapshots, and delegates full durable execution to an engine such as Inngest. For a TypeScript team that is often the better shape anyway: the durable engine is language-agnostic infrastructure, and the AI dependency stays shallow.

If this persona also matches the document back office — long runs, human approval — read that persona too, and treat durability as the deciding requirement rather than language.

Persona: the web team adding one AI feature#

WHO#

Four to eight engineers on an existing Django, Rails, or Node application that has been in production for years. Postgres, a background job queue, a CI pipeline, an on-call rota. Nobody has “AI” in their title. The codebase has conventions and the team likes them.

WHY they are here#

Someone asked for a summarize button, or a support-ticket classifier, or extraction from uploaded PDFs. It is one feature, not a product direction. The team has been told to “use LangChain” by an article, and is trying to work out whether that is right before adding a dependency they will maintain for years.

What actually matters to them#

• The feature must not become an architecture. Adding a graph runtime to a Rails app to summarize a paragraph is a poor trade, and the team knows it.
• On-call. Whatever ships gets paged about. A stack trace that points into framework internals at 3am is a real cost.
• The existing job queue already works. They have Sidekiq or Celery. Retries and background execution are solved problems in this codebase.

Recommendation#

Start with no framework. Provider SDK, a validation library for structured output, and the job queue you already run. If the output shape matters — and for a classifier or an extractor it always does — PydanticAI for Python teams is a small, leaveable step up from raw calls.

Reach for MCP if a tool will be shared by more than one consumer; do not wrap a single internal function in a protocol for one caller.

Why not the obvious answer#

LangChain is the article’s answer, not this team’s. The integration catalog solves a problem they do not have — they have one provider and one internal function. The graph model solves control flow they do not have — one call, one parse. What they would get is a large dependency, a second vocabulary in the codebase, and a deprecation treadmill, in exchange for approximately nothing.

When to revisit#

Revisit when the feature grows a second step that can fail independently, or when somebody asks for a human approval step. Those are the durability triggers from S2, and at that point the honest comparison is LangGraph against the durable engine you may already be running.

The failure mode to avoid#

Adopting the framework “for later”. The requirements that would justify it may never arrive, and if they do, they will arrive in a shape you cannot predict now. Deferring is cheap; unwinding is not.

S3 Recommendation#

The question that sorts every persona#

What happens when the process dies — and does anything wait for a human?

Six personas, and this one question separates them more cleanly than any feature:

What the personas revealed#

Three of six should not adopt an orchestration framework at all. The web team, the solo founder, and the quality-bottleneck team are all better served by the null option or by a tool from a different category. That is not a rhetorical flourish — it is what the analysis produced, and it is the finding most absent from this category’s published comparisons, which are written by and for people who have already decided to adopt one.

The one persona with a short answer is the one with a hard requirement. Durability narrows the field to two options and makes the rest of the comparison moot. Every other persona has a genuine choice, which means the decision is lower-stakes than the volume of comparison content suggests.

The reason someone is shopping predicts the answer better than their stack does. Arriving because runs keep dying leads somewhere completely different from arriving because JSON keeps failing to parse — even though both readers will search the same words and find the same articles.

If none of the personas fits#

Answer these three, in order, and stop when one decides it:

1. Must a run survive a restart or pause for a human? → LangGraph, or a durable engine with the provider SDK. Stop.
2. What language is the application? → TypeScript: Mastra / Vercel AI SDK. .NET or Java: MAF or Semantic Kernel. Python: continue.
3. Is the bottleneck plumbing or quality? → Quality: DSPy, keep your runtime. Plumbing: pick the control-flow shape from S2 that matches your problem, then the library.

If you reach the end without a clear answer, that itself is the answer: your requirements do not yet justify a framework. Ship the plain version and revisit when a second step starts failing independently.

The persona this survey cannot serve#

Anyone who arrives with a specific product already chosen and wants validation. The honest answer to “is LangChain the right choice for us” is that it depends on facts about your system that this survey does not have — and that the question is usually a proxy for “will I be criticised for this decision”, which is not a technical question.

The three questions above are the substitute. They can be answered in an afternoon with facts you already possess.

S4 Approach — Strategic Selection#

Pass: S4 Strategic Re-researched: 2026-08-17

What this pass asks#

S1–S3 answered what to choose today. S4 asks what you are signing up for over a two- to three-year horizon — the period in which a framework choice actually gets tested, and the period over which this category has repeatedly surprised people.

Three questions:

1. convergence-thesis.md — is this category still a category? The strongest finding of the re-research, and the one that should change how the survey is read.
2. vendor-viability.md — who is behind each option, what do they need from it, and what happens to you when their incentives shift.
3. lock-in-and-exit.md — what does each choice cost to leave, measured concretely rather than as a feeling.

Why this pass needed a full re-run#

The previous S4 was written in late 2025 and reasoned about a five-way race between peer frameworks. Within nine months, three of those five changed identity:

• LangChain deprecated its most-used pattern and re-centred on a graph runtime
• Haystack shipped a major version that put agents at the center
• Semantic Kernel was merged into a different product

A strategic pass whose premise is “these five will compete on features” cannot survive that. The premise itself was wrong, and convergence-thesis.md proposes a better one.

Evidence standard for this pass#

Strategic analysis attracts speculation, so this pass restricts itself to:

• Shipped releases and their dates, verified against registries on 2026-08-17
• Stated vendor direction — deprecations, merges, published release policies
• Structural arguments that follow from what shipped

Where a claim about the future is genuinely a guess, it is labeled as one. Trend predictions in this category have a poor record, including in the previous version of this file, which forecast a five-way race that did not occur.

The convergence thesis: this category bifurcated#

The central strategic finding of the 2026 re-research, and the thing a reader should take away even if they remember nothing else.

What happened, in evidence#

Within roughly twelve months, every incumbent in this category re-pointed at agents:

Four independent vendors, four different starting points, one direction. And the convergence is not only strategic but architectural: Google’s ADK arriving independently at a graph of nodes with conditional routing, fan-out/fan-in and built-in human-in-the-loop is the same abstraction LangGraph settled on. The graph-of-nodes model has become the category default rather than one vendor’s opinion.

The wrong conclusion#

That “orchestration frameworks are dead” or “everything is agents now”. Both are the kind of thing this category’s commentary says every six months, and neither survives contact with what shipped. Haystack’s pipelines still exist and got simpler. LangGraph’s deterministic graphs are the substrate the agent stuff is built on. PydanticAI, whose whole proposition is typed plain code, is the most actively released package in the survey.

The right conclusion: it bifurcated#

The category did not die. It split along an axis it did not have in 2025 — who decides the next step — and the two halves now have different vendors, different vocabularies, and different rates of investment.

The model-driven half got the attention, the funding and the branding. It is where every vendor’s 2026 release notes point. It is 1.201.

The deterministic half — what remains this survey’s subject — did not shrink so much as clarify into three distinct products that used to be marketed as one thing:

1. Durable workflow graphs. LangGraph, and general-purpose engines like Temporal and Inngest. The value is that a run survives reality.
2. Compilers. DSPy with GEPA. The value is output quality against a metric, produced at build time.
3. Typed thin layers. PydanticAI, Instructor. The value is that the boundary is checked, and that there is nothing else to learn.

Those three are not competitors. A team can use all three at once — a durable graph, running prompts DSPy compiled, with typed edges — and increasingly the sophisticated answer is exactly that.

And the null option got stronger#

The quiet part. The 2024 case for adopting a framework had three legs and two broke:

• Provider portability — always oversold, and the differences that matter never abstracted cleanly
• Tool integration — expired. MCP became the default tool protocol across the incumbents; the catalog stopped being framework property
• Control flow — intact, and now the only durable argument

So the field’s center of gravity moved toward agents at the same moment the reasons to adopt a general-purpose framework thinned out. Those two facts are usually reported separately. Together they are the strategic picture.

What this means for a decision with a two-year horizon#

Prefer options whose deterministic story is a product, not residue. LangGraph’s durable graphs, Haystack’s declarative pipelines and PydanticAI’s typed calls are things their vendors are selling. A deterministic path that exists mainly because it predates the agent pivot will be maintained and will not be improved.

Expect the graph vocabulary to persist even as libraries change. Nodes, edges, conditional routing, checkpoints, human-in-the-loop: four vendors converged on this independently, which makes it the safest thing to learn in the category. Skills transfer even when the dependency does not.

Assume the tool edge stays a protocol. MCP at the boundary is the most portable commitment available here — more portable than any framework above it.

Treat “which framework” as less consequential than it feels. The decision that persists is the control-flow shape and whether you need durability. Libraries implementing the same shape are substitutable at a cost measured in weeks; a wrong answer on durability is measured in a rewrite.

Falsifiable predictions#

Recorded so the next pass can score them rather than quietly re-forecasting:

1. The deterministic lane will not get major new frameworks. New entrants will be agent-first. Falsified by: a well-funded pipeline-first framework launching.
2. LlamaIndex reaches 1.0 or loses ground. A pre-1.0 line is now the outlier among peers with stability contracts. Falsified by: it staying 0.x while growing share.
3. Durable execution converges on general-purpose engines. More teams will run Temporal or Inngest with a thin LLM layer rather than adopt an LLM-specific durable runtime. Falsified by: LangGraph checkpointing becoming the default even for non-LLM steps.
4. The null option’s share grows, as MCP makes the catalog argument weaker each year. Falsified by: framework adoption accelerating among teams with simple flows.

Lock-in and exit#

Lock-in gets discussed as a feeling. It is measurable: what would you rewrite if this had to go? Ask it before adopting, when the answer is cheap to act on.

The four layers, from portable to sticky#

1. Prompts — fully portable. A prompt is a string. Nothing here owns it, including DSPy, whose entire output is a prompt you keep.

2. Tool integrations — portable if MCP. This is the strategic reason to adopt MCP independent of any framework choice. A tool behind an MCP server is reachable from every framework and from no framework. A tool wrapped in a framework’s own abstraction is reachable from that framework.

3. Control flow — moderately sticky. Rewriting a graph as a pipeline, or a pipeline as plain functions, is work measured in weeks. Recoverable, but not free, and the effort scales with how much of your logic lives in the framework’s shape rather than in your own functions.

4. Persisted state — the sticky layer, and the one nobody prices. Checkpoints written in a framework’s format are the deepest commitment in this survey. Leaving means either draining every in-flight run before switching or writing a migration for a format you do not control. It is the reason LangGraph sits at the bottom of the exit-cost ranking, and the reason a durable engine — whose state format is its documented product — is a defensible alternative.

Practices that keep the exit cheap#

None of these is exotic; all of them are skipped under deadline.

Keep business logic out of nodes. A node should call a function you own. When the framework’s job is scheduling rather than deciding, replacing it is a scheduling change.

Own the state schema. Define what a run’s state contains as your own type, and let the framework carry it. Frameworks that let you hand them an opaque object make this easy; taking their default state shape makes it hard.

Use MCP at the tool edge from day one, even for a single tool, if there is any prospect of a second consumer. This is the highest-leverage portability decision available.

Treat the tracing product as replaceable. Vendor tracing is the most common route from “we use the open-source library” to “we depend on the platform”. See 1.207.

Exit cost, concretely#

The lock-in nobody talks about: your team’s vocabulary#

Six months into a framework, your team thinks in its abstractions. Design discussions happen in nodes and edges, or in components and connections. Job descriptions name it. That is a real switching cost and it does not appear in any comparison table.

It is also an argument for the graph vocabulary specifically: four vendors converged on it independently, so a team that thinks in nodes and edges has learned something that outlives the dependency. Of the sticky things to be stuck with, a widely shared model is the least bad.

The strategic recommendation#

Adopt the portable layers deliberately and the sticky ones reluctantly.

Take MCP at the tool edge — it is the most durable commitment available here, and the one most likely to still be right in three years. Take a compiled-prompt workflow if you have a metric; the artifact is yours. Take a state graph when durability is a genuine requirement, knowing you are accepting the survey’s highest exit cost in exchange for its most substantial capability.

And keep asking the question that started this file. If you cannot say what you would rewrite, the coupling is already deeper than you think.

S4 Recommendation#

The strategic picture in three sentences#

The category bifurcated along who decides the next step, and the model-driven half got the attention, the funding and the branding. What remains here clarified into three non-competing products — durable workflow graphs, prompt compilers, and typed thin layers — which a sophisticated team increasingly uses together. Meanwhile MCP removed the tool catalog as a reason to adopt anything, leaving control flow as the only durable argument for a framework at all.

Choose for a two-year horizon#

1. Prefer a deterministic story that is a product, not residue. LangGraph’s durable graphs, Haystack’s declarative pipelines and PydanticAI’s typed calls are things their vendors sell. A pipeline path that survives mainly because it predates the agent pivot will be maintained, not improved. Every incumbent re-pointed at agents inside twelve months; assume that continues.

2. Learn the graph vocabulary; commit to it more carefully. Nodes, edges, conditional routing, checkpoints and human-in-the-loop are now the shared model — LangGraph, Google ADK Go 2.0 and others converged on it independently. The vocabulary transfers between libraries and is the safest thing to learn here. The checkpoint format does not transfer and is the survey’s highest exit cost.

3. Put MCP at the tool edge regardless of what sits above it. It is the most portable commitment available in this category, and the one most likely to still be correct in three years. Frameworks will keep changing shape around it.

4. Buy durability from whoever sells it best. If durability is your requirement, the choice is LangGraph or a general-purpose engine. If the workflow is mostly model calls, LangGraph keeps it in one place. If it is a business process with a few model calls in it, Temporal or Inngest keeps the AI dependency shallow — and that engine will outlive several generations of LLM framework.

5. Keep the exit priced. Business logic in your own functions; state schema you defined; tracing treated as replaceable. None of it is exotic and all of it is skipped under deadline.

What would change this survey’s advice#

Stated so the next pass has something to check rather than a fresh opinion:

• LlamaIndex reaching 1.0 with a stability contract would remove the main caution against it.
• A serious pipeline-first entrant would falsify the claim that new investment is agent-only.
• MCP fragmenting — competing tool protocols, or vendor extensions that break portability — would restore the catalog argument for frameworks and change the whole strategic picture. Watch 2.074.
• LangSmith becoming practically mandatory for LangGraph operation would move LangGraph from “largest exit cost” to “commercial dependency”, which is a different decision.
• Durable execution appearing natively in Haystack, LlamaIndex or PydanticAI would end LangGraph’s clearest differentiator.

The honest summary for a decision-maker#

For most teams the framework choice is less consequential than it feels and is being made under more marketing pressure than it deserves. The decisions that persist are: whether runs must survive reality, which control-flow shape fits the problem, and whether the tool edge is a protocol or a proprietary wrapper.

Get those three right and the library is substitutable. Get them wrong and no amount of ecosystem breadth compensates.

Vendor viability — who is behind each option, and what they need from it#

Viability in open source is rarely about a project disappearing. It is about what happens to the free library when the entity maintaining it needs revenue, gets acquired, or changes strategy. That question has a different answer for each option here.

The funded startup: LangChain / LangGraph#

Backing: $125M Series B (Sequoia), announced with the 1.0 releases in October 2025. Revenue model: LangSmith — hosted tracing, evaluation and deployment.

Venture funding is not a warning sign; it is what pays for the maintenance and the release policy that this category otherwise lacks. But it sets up a specific dynamic worth naming: the free library is the funnel for the paid platform. The most reliable prediction is that observability and deployment will keep getting easier when you use the vendor’s hosted product.

Watch for: features that are technically open but practically require LangSmith to operate; pricing changes at the hosted layer; and the fate of the OSS release policy if growth targets tighten. None of these is happening now — they are the shape of the risk.

Mitigation: keep your state store yours, and treat tracing as a replaceable component (see 1.207).

The commercial open-core: Haystack / deepset#

Backing: deepset, an enterprise AI company. Revenue model: an enterprise platform built around the open framework.

The structural risk here is the classic open-core one: features migrating from the open framework to the commercial platform over time. The counter-evidence is unusually strong — Haystack 3.0’s migration handling (a dated support window for 2.31, a static scanner shipped as an agent skill) is the behavior of a project that expects to keep its open-source users rather than herd them.

Watch for: the gap between what the open framework does and what the platform does; and whether the ~30 components moved to independent packages keep pace with the core.

The hyperscalers: Microsoft, Google#

Backing: effectively unlimited. Revenue model: cloud consumption.

These will not run out of money, and that is the wrong risk to worry about. The right one is strategic reorganisation — which has already happened once here. Semantic Kernel was the flagship, then AutoGen was the agent story, then both became Microsoft Agent Framework. Nothing was abandoned, and every team that had built on the previous shape did migration work anyway.

Watch for: the next merge. In this category the hyperscalers reorganise their AI framework lines roughly annually, and each reorganisation is announced as continuity.

Mitigation: on these stacks, keep the framework at arm’s length. The direct SDK plus MCP is unusually attractive here precisely because it is not subject to product-line strategy.

The academic project: DSPy#

Backing: Stanford NLP and a research community. Revenue model: none.

A different risk profile entirely. No commercial pressure, no funnel, no acquisition. The risks are the academic ones: maintenance depending on individuals, documentation written for readers who already speak the vocabulary, and priorities following publications rather than production needs.

GEPA reaching ICLR 2026 as an Oral is a genuine signal of health — the work is being reviewed rather than only marketed, which is rare in this field.

Mitigation: the artifact is a prompt. If DSPy stopped tomorrow, the compiled prompts keep working. This is the lowest-lock-in option in the survey.

The independents: PydanticAI, Mastra#

PydanticAI: the Pydantic team, whose core library is a dependency of a large share of the Python ecosystem. Deep credibility and an existing commercial context around Pydantic services. The API stability commitment since 1.0 is the meaningful signal.

Mastra: the team behind Gatsby. Worth remembering that Gatsby’s own commercial arc ended in acquisition and a much-reduced role for the open-source project. That is not a prediction about Mastra; it is the reason to ask what the business model is before making it foundational to a product.

Mitigation for both: these are the smallest-surface options, so the exit is cheap. That is a genuine viability answer, not a consolation.

The unfundable option: no framework#

The null option has no vendor and therefore no vendor risk. Its risks are yours — that you under-build and hit a wall, or over-build and reinvent a worse LangGraph. Both are within your control, which is more than can be said for a strategic reorganisation.

Ranked by “what happens if the maintainer walks away”#

This ranking is not a recommendation to pick the top row. It is the price list, and it should be read alongside what each option does for you — LangGraph sits at the bottom because it does the most.