The minimum implementation of an Agent that can call tools is not complicated. The model receives messages and returns text or a Tool Call; the program executes the tool, appends the Tool Result to the message list, and makes another model request. Vercel AI SDK's ToolLoopAgent encapsulates precisely this kind of multi-step execution: the model, Instructions, Tools, and stopping conditions are centralized in the Agent definition, while the SDK drives the loop from there.[1]
Stripping away this layer of abstraction, the core logic resembles the following code:
This is already a complete Tool Loop. Based on the tool results, the model can continue making decisions, and a single user input can trigger several model calls.
It also has a clear boundary: the entire execution lifecycle is still contained within the current function call.
The messages live in memory, the Tool that is currently executing can be represented by a local variable, cancellation can rely on the current call's AbortSignal, and when the function returns, that execution ends. If the caller does not need to pause, resume, accept input midway, or continue across connections, this design is sufficient.
Agent Runtime usually becomes an independent concern once the execution lifecycle extends beyond the current function call.
This boundary is a better way to judge whether a system genuinely needs a Runtime than a feature checklist such as Tool Calling and Memory.
What a Tool Loop Can Manage
Let's narrow the scope. A Tool Loop manages a local execution chain:
User Input
↓
Model Request
↓
Assistant Message
↓
Tool Call
↓
Tool Execution
↓
Tool Result
↓
Next Model Request
It mainly handles three things:
- when to make a model request;
- how to execute tools when the model calls for them;
- how Tool Results feed into the next model request.
All three occur within a single continuous execution, so local variables and the call stack naturally serve as state containers.
For example, state such as the following can be stored directly in the current invocation:
At this point, there is no need for a more complex object model. If a Runtime merely moves these local variables into a class, it does not automatically acquire any new architectural value.
The real change comes when state begins to have different lifecycles.
The First Change: Execution State Can No Longer Depend on a Request
Web applications are most likely to expose this problem.
Suppose the browser sends a request and the backend starts running the Agent:
Browser Request
│
▼
Agent Loop
│
├── Model
├── Tool
├── Model
└── Tool
If the entire execution must remain tied to the HTTP/SSE connection, a dropped connection usually means the call chain is released. For an ordinary request, that is reasonable: when the client leaves, the request ends.
Agent executions often need different semantics: the connection is responsible only for transport, while the backend continues to manage the task itself. When the client reconnects later, it should read the current state and continue receiving results.
At this point, at least two lifecycles appear:
Connection
──────────────>
Execution
──────────────────────────────>
The connection no longer owns the execution.
Once this split is made, the following state needs an owner independent of the network request:
These pieces of state can no longer be reliably stored in a single Controller invocation or SSE handler.
The Second Change: New Input Can Arrive During Execution
An ordinary function call has one stable premise: the arguments are already fixed when the call begins.
An Agent does not necessarily satisfy this premise. While the current Tool is still running, a user may add requirements; a plugin may inject new environment information; a scheduler may send a follow-up task; another Agent may pass a result to the current Agent.
If the Agent is called directly again while it is running:
The system has to answer a very specific question: which segment of execution does this message belong to?
It may have several entirely different semantics:
These differences can no longer be expressed through a single messages.push().
Input needs to enter a controlled scheduling structure, with the Runtime consuming it at stable boundaries. Pi's Steering / Follow-up Queue and DeepSeek Harness's Inbox handle exactly this layer of the problem.[2][3]
Thus, in addition to executing the Model and Tool, the Runtime begins to take on responsibility for input-delivery semantics.
The Third Change: Separate Long-Term History from Current Execution State
In a Tool Loop, the messages usually play two roles at once:
历史记录
+
下一次模型请求的输入
In a simple implementation, this is not a problem. As the session grows longer, the two gradually separate.
A long-lived Session may retain the full history; what the model can currently see is constrained by the window size, Compaction, Memory, branch selection, and Agent Policy.
Therefore, a more accurate model of the data relationships is:
The Session stores long-term facts; the Context describes the information visible to the current model computation.
If the two continue to share the same messages[], several typical problems emerge:
- Whether Compaction overwrites the original history;
- Whether the UI can still display messages that were compacted away;
- Whether the same Session can use different Context Policies when passed to different Agents;
- Whether temporary state from Tool execution should enter the long-term history;
- Whether Provider Message and the application's internal Message must use the same structure.
Runtime therefore also has to be responsible for constructing the current computational view from long-term state.
The Core of Runtime Is State Ownership
At this point, we can return to the Tool Loop we started with.
Originally, all state was implicitly owned by the current function:
function call
├── messages
├── tool state
├── abort signal
└── control flow
Once the lifecycles are separated, ownership of state must be reassigned:
The value of Runtime thus becomes concrete: it provides a stable owner for execution state that spans a single function call and drives that state forward.
A minimal structure can be represented as:
In this diagram, the Loop is just one component of the Runtime. The Runtime also has to maintain state and control relationships outside the Loop.
Runtime Also Needs to Distinguish Configuration, Active State, and Durable Facts
Once execution state has been pulled out of the function call, another question follows: what should persist long term, and what is valid only during the current activity.
The data around the Runtime can be divided into three categories:
These three types of data are often placed in the same Agent object, but their lifecycles differ.
AgentDefinition can be reused by multiple Sessions and may persist long term under a version number. It describes “what capabilities this Agent has.”
Live Runtime State describes only what a given process is doing at the moment. AbortController, active promises, and network stream handles all belong to this layer. Once the process ends, these objects have no direct value for restoration.
Durable Session State has to be saved across processes. It should record enough facts for a new Runtime to reconstruct a view of the current work.
Therefore, when restoring an Agent, the more stable path is to create a new Runtime from durable facts rather than directly deserializing old in-process objects:
Durable Session Facts
+
Agent Definition / Version
↓
create new Runtime
↓
rebuild Context / pending work
↓
continue
This boundary directly affects the persistence design. Serializing every in-memory field often yields only a Runtime Snapshot that is difficult to upgrade and hard to restore across processes. A more stable approach is to persist the facts needed to rebuild the runtime state, then have the new Runtime create the in-process objects.
From this perspective, Runtime and Persistence are adjacent layers: Runtime manages active state, while Persistence stores the facts needed for recovery. The two can work closely together without sharing the same set of objects.
Runtime Is More Like a State Machine Than a Set of Tool Functions
Once the Runtime owns states such as idle / running / waiting / cancelled, the validity of many APIs is determined by the current state as well.
For example, Pi does not allow another prompt() while active, but permits steer(), followUp(), and abort(); when idle, it can begin a new Prompt.
The model can be abstracted as:
The Runtime therefore not only stores data but also constrains which state transitions are valid.
This matters for API design. If the external interface is merely a set of stateless functions, the caller has to understand for itself whether a current execution exists, where an input should go, and when cancellation is possible. Bringing these constraints into the Runtime surfaces invalid states earlier and gives higher-level products a more stable behavioral contract.
Why Pi Makes a Good Runtime Example
Pi's Agent is clearly separated from the underlying agent-loop.ts.
The low-level Loop drives model requests, Tool Calls, Tool Results, Steering, and Follow-up; the outer Agent is described in the source as a stateful wrapper around the low-level agent loop.[2]
The state held by the Agent roughly includes:
These fields form a clear hierarchy:
messages
已经完成的稳定 transcript
streamingMessage
当前正在生成、仍会变化的 Assistant Message
pendingToolCalls
当前执行中的 Tool 状态
This state is not all packed into messages.
Pi also maintains a Steering Queue and a Follow-up Queue. While the Agent is active, it prevents another direct prompt() from creating a second proactive execution chain; new messages must be handed to the current Runtime through explicit input-control APIs.[2]
This already matches the Runtime characteristics derived above: persistent state, a single active execution chain, input scheduling, cancellation, and lifecycle events all have clear owners.
DeepSeek Harness Abstracts Runtime One Layer Further
DeepSeek Harness takes the Agent abstraction one layer further, toward the Harness.
The public Agent handle exposes:
session
inbox
status
ctx
cancel()
whenIdle()
send()
steer()
followup()
inject()
The concrete Agent Loop is supplied by plugins.[3]
Here, another important change appears: the application and plugins depend on a stable Agent interface, while the concrete Loop implementation can be replaced.
This means the Runtime's “execution mechanism” itself can be replaced. Agent Loop, Session, Tool Registry, and Model Adapter do not all have to be locked into a single core class. This direction eventually leads into Harness and Plugin Runtime; a later Cordis / DeepSeek Harness unit in this series will address it separately.
In the first article, we only need to retain one distinction:
Runtime addresses the lifecycle and progression of execution state; Harness further addresses how Runtime capabilities are composed, replaced, and extended.
The two layers have different concerns.
Vercel AI SDK Provides a Useful Reference from the Other Side
Its value lies in separating several layers that web applications often conflate.
ToolLoopAgent handles multi-step Agent execution; AI SDK UI provides useChat, Transport, UI Message, message persistence, and Stream Resume.[1][6]
It can therefore be understood through the following diagram:
Browser UI
│
▼
Transport
│
▼
Application / Agent Execution
│
▼
Model + Tools
AI SDK does not require every product to adopt a long-lived Agent object like Pi or DSH, but it offers a clear point of reference: UI state, network transport, Agent execution, and model messages can be separate layers.
This is an important foundation for the later discussion of Web Agent Runtime.
The Boundary Between Runtime and Agent Framework Also Needs Control
Once Runtime is independent, it becomes easy to fold every Agent capability into this layer, recreating an ever-expanding core.
A useful initial boundary is whether a capability directly participates in advancing the current execution.
The following capabilities are usually tightly coupled to Runtime:
Context preparation
Model invocation
Tool execution
Input delivery
Cancellation
Lifecycle state
They directly affect what the current Loop does next.
The following capabilities are more likely to belong to the Harness or Application layer:
Plugin discovery
Skill marketplace
Project management
Workflow definition
UI layout
Billing
Tenant permission
They can register a Tool, Hook, or Policy with the Runtime, but they do not need to become intrinsic fields of the low-level execution object.
Pi and DSH can serve as a contrast between two extension boundaries. Pi tends to keep a small Agent Core and extend its behavior through Extensions; DSH goes further, moving capabilities such as Agent Loop, Session, and Tool Registry themselves into the Plugin Runtime. The extension boundaries differ, but both address the same problem: how to keep Agent Core from expanding indefinitely as product capabilities grow.
Therefore, the Runtime abstraction should also remain restrained. It needs to own the execution lifecycle completely enough while leaving room for the outer Harness to compose capabilities.
A Practical Method for Checking State Ownership
When designing a new field or capability, ask four questions in succession:
For example, pendingToolCalls is valid only during the current execution and directly affects the UI and cancellation controls, so the Runtime is a good owner.
Session Branch has to be recoverable across processes and has no direct control role in the current Tool call, so it is better placed in Session Persistence.
Run Status needs to be queried and recovered independently by the product, so it can live in the Application Runtime.
This method makes it easier to maintain clear boundaries than putting every Agent-related feature into the Agent class.
Why Session, Run, Turn, and Step Also Appear in Runtime
Once the execution lifecycle is separated out, several different time scales still exist:
They solve problems at different scopes.
The Session manages the long-term conversation; the Run often represents one durable execution at the application layer; a Turn represents a span of continuous Agent work; and a Step is close to one model decision and the Tool work directly produced by it.
These objects are not there to make the architecture more complicated. Each time a lifecycle is added, it is usually because the layer above cannot stably express some state.
The second article will continue this derivation and establish the boundaries among Session, Run, Turn, and Step.
Why Context Belongs in the Runtime Chain
Before starting a Loop, Pi calls createContextSnapshot():
It then passes this AgentContext to runAgentLoop().[5]
Before actually reaching the Provider, the Context can still pass through transformContext and convertToLlm.
This implementation reveals an easily overlooked boundary:
Long-lived Agent State
│
▼
Context Snapshot
│
▼
Loop Local Context
│
▼
Provider Context
The model input is a computational view constructed by the Runtime at a particular point in time.
This is why, when adding Session persistence later, we cannot simply treat “what the model currently sees” and “what the system stores long term” as the same question.
How to Tell Whether a System Really Needs an Agent Runtime
Not every Agent application needs a complex Runtime.
If the system meets the following conditions:
一次请求内完成
没有中途输入
没有暂停和恢复
不要求跨连接继续执行
消息列表直接作为模型上下文
Tool 状态不需要独立暴露
A Tool Loop plus a small amount of state management is usually enough.
As the requirements grow into the following:
Only then does the Runtime have a clear value as an independent layer.
The deciding criterion should be whether the state lifecycles have split. Framework selection itself cannot replace that judgment.
The Boundaries Used in This Series
Subsequent articles will consistently use the following relationships:
where:
- AgentDefinition describes the Agent's configuration and capability set;
- Session stores long-term conversation facts;
- Runtime manages the current execution state and advances state changes;
- Run provides a management boundary for one application-level execution;
- Turn and Step describe boundaries for continuous work and model decisions within the Loop;
- Context is constructed from the current state before a model request.
This model is used in later articles to compare the implementation boundaries of different frameworks; it does not require them to adopt the same API.
The first article ultimately needs to leave one conclusion:
Once an Agent's execution lifecycle begins to detach from a function call or a network request, state ownership must be redivided. Tool Loop remains the execution kernel, but Runtime begins to emerge as an independent system layer.
The next article will turn to the time scales of these states, explaining why Session, Run, Turn, and Step need to exist separately, and which boundaries should remain in Agent Core and which are better left to the application layer.
References
[1] Vercel AI SDK, ToolLoopAgent: https://ai-sdk.dev/docs/reference/ai-sdk-core/tool-loop-agent
[2] Pi Agent source code: https://github.com/badlogic/pi-mono/blob/main/packages/agent/src/agent.ts
[3] DeepSeek Harness core subsystem: https://github.com/deepseek-ai/deepseek-harness/blob/master/docs/subsystems/core.md
[4] DeepSeek Harness Agent lifecycle: https://github.com/deepseek-ai/deepseek-harness/blob/master/docs/agent-lifecycle.md
[5] Pi createContextSnapshot(): https://github.com/badlogic/pi-mono/blob/main/packages/agent/src/agent.ts
[6] Vercel AI SDK UI: https://ai-sdk.dev/docs/ai-sdk-ui
[7] Pi Coding Agent README: https://github.com/badlogic/pi-mono/blob/main/packages/coding-agent/README.md
[8] DeepSeek Harness architecture: https://github.com/deepseek-ai/deepseek-harness/blob/master/docs/architecture.md