A tool-calling agent's minimal implementation is straightforward. The model receives messages and returns text or a Tool Call; the program executes the tool, appends the Tool Result to the message list, and continues requesting the model. Vercel AI SDK's ToolLoopAgent encapsulates this multi-step execution pattern: the model, instructions, tools, and stop conditions are centralized within the Agent definition, with the SDK driving the subsequent loop.[1]
Once you strip away this abstraction, the core logic resembles this code:
while (true) {
const response = await model(messages, tools)
messages.push(response.message)
if (response.toolCalls.length === 0) {
break
}
const results = await executeTools(response.toolCalls)
messages.push(...results)
}
This is already a complete Tool Loop. The model can continue making decisions based on tool results, and a single user input can trigger multiple model calls.
It also has a clear boundary: the entire execution lifecycle remains contained within the current function call.
Messages exist in memory, the currently executing Tool can be expressed through local variables, cancellation can rely on the current call's AbortSignal, and when the function returns, the execution ends. If the caller doesn't require pause, resume, mid-execution input, or continuation across connections, this structure is sufficient.
The Agent Runtime typically becomes a distinct problem when the execution lifecycle exceeds this single function call.
This boundary is better suited than tool calling or memory capability checklists for determining whether a system truly needs a Runtime.
What Tool Loop Can Manage
First, let's constrain the problem scope. Tool Loop manages a local execution chain:
User Input
↓
Model Request
↓
Assistant Message
↓
Tool Call
↓
Tool Execution
↓
Tool Result
↓
Next Model Request
It primarily handles three things:
- When to request the model;
- How to execute tools when the model requests them;
- How Tool Results return to the next model request.
All three happen within a single continuous execution, so local variables and the function stack naturally serve as state containers.
For example, the following states can be directly stored in the current call:
let messages: Message[]
let pendingToolCalls: ToolCall[]
let aborted = false
At this point, there's no need to introduce a more complex object model.所谓 Runtime，如果只是把这些局部变量挪进一个 Class，并不会自动获得新的架构价值。The real change comes when states begin to have different lifecycles.
First Change: Execution State Can No Longer Depend on Requests
Web applications most easily expose this issue.
Assume a browser initiates a request and the backend starts running an Agent:
Browser Request
│
▼
Agent Loop
│
├── Model
├── Tool
├── Model
└── Tool
If the entire execution must depend on the HTTP/SSE connection, then connection termination typically means the call chain is released. For normal requests this is reasonable: the client leaves, the request ends.
Agent execution often needs a different semantics: the connection is only responsible for transmission, while the task itself continues to be managed by the backend. When a client reconnects later, it should read the current state and continue receiving results.
At this point, at least two lifecycles emerge:
Connection
──────────────>
Execution
──────────────────────────────>
The connection no longer owns the execution.
Once this split is made, the following states must have owners independent of network requests:
Whether execution is still in progress
Which stage of execution has been reached
Which tools are currently running
Whether a cancellation request has been received
Where subsequent input is waiting
Whether the final result has been produced
These states can no longer be reliably stored in a single Controller call or SSE handler.
Second Change: New Input Arrives During Execution
Normal function calls have a stable premise: parameters are determined when the call starts.
Agents don't necessarily satisfy this premise. While the current Tool hasn't finished executing, the user might add requirements; plugins might inject new environment information; schedulers might send follow-up tasks; other agents might also pass results to the current agent.
If the Agent is invoked again directly while running:
agent.prompt(newMessage)
The system must answer a very specific question: which execution segment does this message belong to?
It could have several completely different semantics:
Immediately terminate current work and start over
Wait for the current Tool to complete, affecting the next model decision
Wait for the current Turn to finish completely, then start the next piece of work
Just add to the next Context, but don't actively trigger execution
These differences can no longer be expressed through a single messages.push().
Input needs to enter a controlled scheduling structure, consumed by the Runtime at stable boundaries. Pi's Steering / Follow-up Queue, and DeepSeek Harness's Inbox, handle this layer of the problem.[2][3]
Therefore, in addition to executing Model and Tool, Runtime begins to bear input delivery semantics.
Third Change: Long-term History Separates from Current Execution State
Messages in Tool Loop typically play two roles simultaneously:
History record
+
Input for next model request
In simple implementations, this works fine. As conversations grow longer, the two gradually separate.
A long-term Session might retain complete history; what the model currently sees is affected by window size, Compaction, Memory, branch selection, and Agent Policy.
Therefore, a more accurate data relationship is:
Session History
│
├── Branch Selection
├── Compaction
├── Memory
├── Agent Policy
└── Runtime State
│
▼
Context Builder
│
▼
Model Context
Session saves long-term facts, while Context describes what information is visible to one model computation.
If these two concepts continue to share the same messages[], several typical problems emerge afterward:
- Whether Compaction overwrites the original history;
- Whether the UI can still display messages that were compacted away;
- Whether the same Session can use different Context Policies when handed to different Agents;
- Whether temporary state during Tool execution should enter long-term history;
- Whether Provider Message and application-internal Message must use the same structure.
Therefore, Runtime also needs to be responsible for constructing the current computation view from long-term state.
The Core of Runtime is State Ownership
At this point, we can revisit the initial Tool Loop.
Originally, all state was implicitly owned by the current function:
function call
├── messages
├── tool state
├── abort signal
└── control flow
After splitting the lifecycle, state ownership needs to be redistributed:
Session
└── Long-term history, session metadata
Agent Runtime
├── Current execution state
├── Context construction
├── Tool Runtime
├── Input Queue / Inbox
├── Cancellation
└── Lifecycle Events
Connection
└── Real-time transmission and reconnection
Runtime's value becomes concrete here: it provides a stable owner for execution states that span a single function call, and is responsible for advancing these states.
A minimal structure can be represented as:
Agent Runtime
│
┌──────────────┼──────────────┐
│ │ │
▼ ▼ ▼
Model Tool Runtime Input Delivery
│ │ │
└──────────────┼──────────────┘
▼
Agent Loop
│
┌──────┴──────┐
▼ ▼
Context Lifecycle Event
In this diagram, Loop is just one component of Runtime. Runtime also needs to maintain state and control relationships outside the Loop.
Runtime Also Needs to Distinguish Configuration, Active State, and Durable Facts
After extracting execution state from function calls, another question emerges: which content should persist long-term, and which is only valid during current activity.
Data around Runtime can be divided into three categories:
Agent Definition
Describes capability configuration
Model / Prompt / Tools / Policy
Live Runtime State
Describes current activity
streaming / pending tool / inbox / abort
Durable Session State
Describes recoverable facts
message / event / branch / metadata
These three categories are often placed in the same Agent object, but their lifecycles differ.
Agent Definition can be reused by multiple Sessions and may persist long-term through version numbers. It describes "what capabilities this Agent has."
Live Runtime State only describes what's happening in a certain process right now. AbortController, active promises, and network stream handles belong to this layer. After the process ends, these objects have no direct recovery value.
Durable Session State needs to persist across processes. It should record enough facts to allow a new Runtime to reconstruct the current work view.
Therefore, when recovering an Agent, a more stable path is to create a new Runtime based on persistent facts, rather than directly deserializing old in-process objects:
Durable Session Facts
+
Agent Definition / Version
↓
create new Runtime
↓
rebuild Context / pending work
↓
continue
This boundary directly affects persistence design. Serializing all memory fields often only yields a Runtime Snapshot that's difficult to upgrade and hard to recover across processes. A more stable approach is to save facts that can rebuild the running state, then have the new Runtime create in-process objects.
From this perspective, Runtime and Persistence are adjacent layers: Runtime manages active state, while Persistence is responsible for saving facts needed for recovery. The two can collaborate closely but don't need to share the same object set.
Runtime is Closer to a State Machine Than a Set of Utility Functions
When Runtime has states like idle / running / waiting / cancelled, the legality of many APIs is also determined by the current state.
For example, Pi prohibits calling prompt() again in the active state, allowing steer(), followUp(), and abort(); in the idle state, a new Prompt can be started.
This can be abstracted as:
prompt
IDLE ─────────────→ ACTIVE
│ │
│ ├── steer / follow-up
│ └── abort
│
└────────────→ IDLE
finish
Therefore, Runtime not only stores data but also constrains which state transitions are legal.
This is important for API design. If external interfaces are just several stateless functions, callers must themselves understand whether current execution exists, where a piece of input should go, and when cancellation is possible. Bringing these constraints into Runtime can expose error states earlier and give上层产品更稳定的行为契约。
Why Pi is Suitable as a Runtime Sample
Pi's Agent is fairly clearly separated from the underlying agent-loop.ts.
The low-level Loop is responsible for model requests, Tool Call, Tool Result, Steering, and Follow-up execution advancement; the outer Agent is described in source code as a stateful wrapper around the low-level agent loop.[2]
The states held by Agent roughly include:
systemPrompt
model
thinkingLevel
tools
messages
isStreaming
streamingMessage
pendingToolCalls
errorMessage
These fields form a clear layering:
messages
Stable transcript that has been completed
streamingMessage
Currently being generated, still changing Assistant Message
pendingToolCalls
Tool state currently in execution
These states aren't all stuffed into messages.
Pi also maintains Steering Queue and Follow-up Queue. During Agent activity, directly calling prompt() to create a second active execution chain is prohibited; new messages need to be handed to the current Runtime through explicit input control APIs.[2]
This already matches the Runtime characteristics derived earlier: persistent state, single active execution chain, input scheduling, cancellation and lifecycle events have clear owners.
DeepSeek Harness Abstracts Runtime Outward One More Layer
DeepSeek Harness's abstraction of Agent leans more toward Harness.
The public Agent handle exposes:
session
inbox
status
ctx
cancel()
whenIdle()
send()
steer()
followup()
inject()
The specific Agent Loop is provided by plugins.[3]
Another important change appears here: applications and plugins depend on a stable Agent interface, while specific Loop implementations can be replaced.
This means the "execution mechanism" of Runtime itself can also be replaced. Agent Loop, Session, Tool Registry, and Model Adapter don't need to all be solidified in one core class. This direction eventually enters Harness and Plugin Runtime; the Cordis / DeepSeek Harness units later in this series will handle this separately.
In the first article, only one judgment needs to be retained:
Runtime solves the lifecycle and advancement problems of execution state; Harness further solves how Runtime capabilities can be combined, replaced, and extended.
The two layers have different concerns.
Vercel AI SDK Provides Another Point of Reference
Vercel AI SDK's value lies in separating several layers that are often mixed together in Web applications.
ToolLoopAgent is responsible for multi-step Agent execution; AI SDK UI provides capabilities like useChat, Transport, UI Message, message persistence, and Stream Resume.[1][6]
Therefore, it can be understood in the context of this diagram:
Browser UI
│
▼
Transport
│
▼
Application / Agent Execution
│
▼
Model + Tools
AI SDK doesn't require all products to adopt long-lifecycle Agent objects like Pi or DSH, but it provides a clear reference: UI state, network transmission, Agent execution, and model messages can be different layers.
This is also an important foundation for discussing Web Agent Runtime later.
The Boundary Between Runtime and Agent Framework Also Needs Control
After Runtime becomes independent, it's easy to continue attributing all Agent capabilities to this layer. This would reform an ever-expanding core.
Use "whether it directly participates in current execution advancement" as a preliminary boundary.
The following capabilities are usually strongly related to Runtime:
Context preparation
Model invocation
Tool execution
Input delivery
Cancellation
Lifecycle state
They directly affect how the next step in the current Loop runs.
The following capabilities are more likely located at the Harness or Application layer:
Plugin discovery
Skill marketplace
Project management
Workflow definition
UI layout
Billing
Tenant permission
These capabilities can register Tools, Hooks, or Policies with Runtime, but there's no need for them to become固有字段 of lower-level execution objects.
Pi and DSH can serve as two examples of expanding boundaries. Pi tends to keep a smaller Agent Core and extend behavior through Extensions; DSH further allows Agent Loop, Session, Tool Registry, and other capabilities to enter Plugin Runtime. The two solutions have different expansion boundaries, but both are addressing the same problem: how to prevent Agent Core from continuously expanding as product capabilities grow.
Therefore, Runtime's abstraction should also remain restrained. It needs to fully own the execution lifecycle while leaving combination space for the outer Harness layer.
A Practical State Ownership Check Method
When designing new fields or capabilities, you can check four questions in sequence:
Does this state span processes?
Is this state only valid during current execution?
Does the model's next decision directly depend on it?
Does an external system need to independently query or recover it?
For example, pendingToolCalls is only valid during current execution and directly affects UI and cancellation control, making it suitable for Runtime to hold.
Session Branch needs to recover across processes and has no direct control over a single Tool call, making it more suitable for Session Persistence.
Run Status needs to be independently queried and recovered by the product and can be placed in Application Runtime.
This check method is clearer for maintaining boundaries than "put all Agent-related functionality into the Agent class."
Why Session, Run, Turn, and Step Still Appear in Runtime
After the execution lifecycle is separated, multiple different time scales still exist:
Session
────────────────────────────────────────>
Run
────────────────────>
Turn
─────────────>
Step
──────>
Model Call
───>
They each solve problems at different scopes.
Session manages long-term conversation; Run is often used for one persistent execution at the application layer; Turn represents a continuous piece of Agent work; Step is close to one model decision and the Tool work directly produced by it.
These objects aren't just for making the architecture more complex. Each additional lifecycle layer usually appears because the previous layer couldn't stably express some state.
For example:
Model Call ended
but Tool Loop hasn't ended
→ Need a larger execution boundary
Turn ended
but Conversation should continue
→ Need Session
Page disconnected
but backend task should continue
→ Separate Connection from Run
The second article will continue along this derivation, establishing the boundaries of Session, Run, Turn, and Step, and explaining which boundaries should remain in Agent Core and which are better suited for the application layer.
Why Context Belongs to the Runtime Chain
Pi calls createContextSnapshot() before starting a Loop:
return {
systemPrompt: this._state.systemPrompt,
messages: this._state.messages.slice(),
tools: this._state.tools.slice(),
}
Then passes this AgentContext to runAgentLoop().[5]
Before actually entering the Provider, Context can also go through transformContext and convertToLlm.
This implementation reveals an easily overlooked boundary:
Long-lived Agent State
│
▼
Context Snapshot
│
▼
Loop Local Context
│
▼
Provider Context
Model input is a computation view constructed by Runtime at a certain point in time.
This is also why, when doing Session persistence later, "what the model currently sees" and "what the system saves long-term" cannot simply be treated as the same problem.
How to Determine if a System Really Needs Agent Runtime
Not all Agent applications need complex Runtime.
If the system satisfies these conditions:
Completes within one request
No mid-execution input
No pause and resume
Doesn't require continuation across connections
Message list directly serves as model context
Tool state doesn't need independent exposure
A Tool Loop with a small amount of state management is usually sufficient.
When requirements gradually become:
Execution spans network connections
Execution can be paused, cancelled, and resumed
User can append input during execution
Session spans multiple executions
Context needs independent construction
UI needs to observe streaming and Tool state
Runtime has clear independent value.
The judgment basis should be placed on whether the state lifecycle has split; framework selection itself cannot replace this judgment.
The Boundary Used in This Series
Subsequent articles uniformly use this set of relationships:
AgentDefinition
/ | \
Model Tools Policy
\ | /
\ | /
Runtime
│
Session ────────────────┤
│
Run
│
Turn
│
Step
│
┌─────────┴─────────┐
▼ ▼
Model Call Tool Execution
Among them:
- AgentDefinition describes the Agent's configuration and capability set;
- Session saves long-term conversation facts;
- Runtime manages current execution state and advances state changes;
- Run provides a management boundary for one application-layer execution;
- Turn and Step describe continuous work and model decision boundaries within the Loop;
- Context is constructed from current state before model requests.
This model is used for subsequent comparison of different frameworks' implementation boundaries, without requiring them to use the same API.
The first article only needs to leave one judgment:
When an Agent's execution lifecycle begins to detach from a single function call or network request, state ownership must be re-divided. Tool Loop remains the execution kernel, but Runtime begins to become an independent system layer.
The next article continues processing these state time scales, explaining why Session, Run, Turn, and Step need to exist separately, and which boundaries should remain in Agent Core versus which are better suited for the application layer.
References
[1] Vercel AI SDK, ToolLoopAgent: https://ai-sdk.dev/docs/reference/ai-sdk-core/tool-loop-agent
[2] Pi Agent source: https://github.com/badlogic/pi-mono/blob/main/packages/agent/src/agent.ts
[3] DeepSeek Harness core subsystem: https://github.com/deepseek-ai/deepseek-harness/blob/master/docs/subsystems/core.md
[4] DeepSeek Harness agent lifecycle: https://github.com/deepseek-ai/deepseek-harness/blob/master/docs/agent-lifecycle.md
[5] Pi createContextSnapshot(): https://github.com/badlogic/pi-mono/blob/main/packages/agent/src/agent.ts
[6] Vercel AI SDK UI: https://ai-sdk.dev/docs/ai-sdk-ui
[7] Pi Coding Agent README: https://github.com/badlogic/pi-mono/blob/main/packages/coding-agent/README.md
[8] DeepSeek Harness architecture: https://github.com/deepseek-ai/deepseek-harness/blob/master/docs/architecture.md