See Part 0 for the latest table of contents and sample code. New chapters will be added over time.

Chapter 8: The Agent Loop and Session Continuity

Chapter 7 gave abcb one clean step. The framework could ask the provider for one assistant message, parse that message as ModelOutput, and either return a final answer or execute one tool.

That was the first time the model's structured decision crossed into framework action. But it was still not an agent loop.

This post is also available on Medium. If you’re a paid Medium member and happen to read it there, it helps fund my next cup of coffee. Much appreciated ☕️😄

If the model asks for a tool, the tool result is not usually the final answer. It is new information. The model needs to see that information and decide what to do next. Maybe it can now answer the user. Maybe it needs another tool. Maybe it made a poor choice and needs to recover. Whatever happens next, the mechanism is the same: the framework must add the result back into the conversation and call the provider again.

That is what this chapter adds.

The loop is small, but it changes the shape of the program. A user message starts a session. The model returns a structured envelope. A tool call can produce a tool result. That tool result becomes a Role::Tool message in the same session. Then the next provider call sees the updated session.

That last sentence is the heart of the chapter. Session continuity is the agent's memory mechanism for this loop. The model does not remember our Rust values. The framework has to carry the conversation forward.

The sample code for this chapter is in chapter08/abcb/.

8.1 The Loop Shape

The agent loop is the cycle we have been circling around:

The important part is the arrow from tool result back to session. Without that arrow, the loop can execute a tool but the model never sees the result. It would be like asking someone to look something up, hiding the answer from them, and then asking them to continue.

Chapter 7 ended with StepOutcome::ToolExecuted. Chapter 8 turns that outcome into continuity. A tool result is no longer only something returned to Rust. It is also appended to the conversation as a tool message.

8.2 run_step Borrows the Session

The first change is that run_step no longer creates a new session by itself. It receives a mutable borrow of an existing session:

File: abcb/crates/abcb-core/src/lib.rs

pub fn run_step(
    provider: &mut impl Provider,
    registry: &ToolRegistry,
    session: &mut Session,
) -> Result<StepOutcome, LoopError> {
    let reply = provider.complete(session)?;
    session.push_message(Message::new(Role::Assistant, reply.content.clone()));

    let output = ModelOutput::parse(&reply.content)?;

    match output {
        ModelOutput::Final { content } => Ok(StepOutcome::Final(content)),
        ModelOutput::ToolCall {
            tool_name,
            arguments,
            ..
        } => {
            let tool = registry
                .get(&tool_name)
                .ok_or_else(|| LoopError::UnknownTool(tool_name.clone()))?;
            let output = tool.invoke(&arguments)?;
            session.push_message(Message::new(Role::Tool, output.clone()));
            Ok(StepOutcome::ToolExecuted { tool_name, output })
        }
    }
}

The signature tells the story:

session: &mut Session

run_step does not own the session. The loop owns it. run_step only borrows it long enough to add messages and ask the provider for a response.

That is the right ownership shape for a loop. If each step created its own fresh session, the model would only see the current user message. The tool result from the previous step would disappear. If each step took ownership of the session and returned it, the code would be noisier than necessary. The loop already owns the session, so a mutable borrow is enough.

The borrow also expresses intent. run_step is allowed to mutate the session, but it is not allowed to replace the session wholesale or keep it after the function returns.

8.3 Recording the Assistant Message

The first mutation happens immediately after the provider returns:

File: abcb/crates/abcb-core/src/lib.rs

let reply = provider.complete(session)?;
session.push_message(Message::new(Role::Assistant, reply.content.clone()));
let output = ModelOutput::parse(&reply.content)?;

The first line calls the provider with the current session. The provider trait still receives the session as a shared reference:

fn complete(&mut self, session: &Session) -> Result<Message, ProviderError>;

That means the provider can read the conversation context, but it does not mutate the session directly. Mutation belongs to run_step. Once the provider returns a reply, run_step appends that reply as an assistant message:

session.push_message(Message::new(Role::Assistant, reply.content.clone()));

This means the session records what the model actually returned, not only what the framework later decided to do with it.

There is a small clone here:

reply.content.clone()

We clone the content because the session needs to own the assistant message, but the parser still needs to read the original reply content afterward:

session.push_message(Message::new(Role::Assistant, reply.content.clone()));
let output = ModelOutput::parse(&reply.content)?;

If we moved reply.content into the session without cloning it, we could not use reply.content on the next line. The clone keeps the sequence simple: the session owns one copy, and reply.content remains available for parsing.

One subtle detail: the assistant message is appended before parsing. If the model returns malformed JSON, the session still contains the model's raw reply. Chapter 9 will build recovery behavior around errors, but even here the ordering is meaningful. The framework records what the provider said before deciding whether it can understand it.

8.4 The Tool Result Becomes a Tool Message

The second mutation happens only on the tool-call path:

File: abcb/crates/abcb-core/src/lib.rs

let output = tool.invoke(&arguments)?;
session.push_message(Message::new(Role::Tool, output.clone()));
Ok(StepOutcome::ToolExecuted { tool_name, output })

This is where Role::Tool finally earns its place in the Role enum.

Back in Chapter 3, Role::Tool looked a little premature. We had messages, but no tools. Then Chapter 6 gave us tools, and Chapter 7 let the model request one. This chapter closes the loop: after the tool runs, its output becomes a message in the session.

The model can see it on the next provider call because run_loop will call run_step again with the same session.

File: abcb/crates/abcb-core/src/lib.rs

for _ in 0..max_steps {
    match run_step(provider, registry, &mut session)? {
        StepOutcome::Final(answer) => return Ok(answer),
        StepOutcome::ToolExecuted { .. } => {}
    }
}

The same session is borrowed each time through the loop. So when one step appends a tool message, the next step passes a session that already contains that tool result.

That is a small line of code with large consequences:

session.push_message(Message::new(Role::Tool, output.clone()));

Without it, the tool output would exist only as a Rust return value, and the next provider call would have no new information from the tool.

The test makes this visible:

File: abcb/crates/abcb-core/src/lib.rs

#[test]
fn run_step_appends_assistant_and_tool_messages_to_session() {
    let mut provider = MockProvider::new([
        r#"{"kind":"tool_call","tool_name":"stub_echo","arguments":{"text":"pong"}}"#,
    ]);
    let registry = registry_with_stub_echo();
    let mut session = session_with_user("hi");

    run_step(&mut provider, &registry, &mut session).expect("step should succeed");

    // user + assistant (the tool call) + tool (the result)
    assert_eq!(session.messages.len(), 3);
    assert_eq!(session.messages[1].role, Role::Assistant);
    assert_eq!(session.messages[2].role, Role::Tool);
    assert_eq!(session.messages[2].content, "pong");
}

This is not only checking a vector length. It is checking the mechanism that makes the loop continuous.

8.5 run_loop

Now the loop itself:

File: abcb/crates/abcb-core/src/lib.rs

pub fn run_loop(
    provider: &mut impl Provider,
    registry: &ToolRegistry,
    user_message: impl Into<String>,
    max_steps: usize,
) -> Result<String, LoopError> {
    let mut session = Session::new("run-loop");
    session.push_message(Message::new(Role::User, user_message));

    for _ in 0..max_steps {
        match run_step(provider, registry, &mut session)? {
            StepOutcome::Final(answer) => return Ok(answer),
            StepOutcome::ToolExecuted { .. } => {}
        }
    }

    Err(LoopError::MaxStepsExceeded { max_steps })
}

This is the first real agent loop in the book.

The code begins by creating the session and seeding it with the user's message:

let mut session = Session::new("run-loop");
session.push_message(Message::new(Role::User, user_message));

That makes run_loop the owner of the session for now. After that, each iteration borrows the same session mutably:

run_step(provider, registry, &mut session)?

That repeated mutable borrow is how history accumulates. The session is not moved into run_step. It stays in the loop, and each step gets temporary permission to mutate it.

Each step can end the loop with a final answer:

StepOutcome::Final(answer) => return Ok(answer),

Or it can execute a tool and let the loop continue:

StepOutcome::ToolExecuted { .. } => {}

The empty ToolExecuted arm works because run_step already appended the tool result to the session. There is no extra loop-side work to do before the next iteration.

8.6 Loop Control Flow

The control flow is plain Rust:

File: abcb/crates/abcb-core/src/lib.rs

for _ in 0..max_steps {
    match run_step(provider, registry, &mut session)? {
        StepOutcome::Final(answer) => return Ok(answer),
        StepOutcome::ToolExecuted { .. } => {}
    }
}

Err(LoopError::MaxStepsExceeded { max_steps })

The range 0..max_steps runs at most max_steps times. If max_steps is 5, the loop has five chances to reach a final answer.

Rust ranges

0..max_steps is a half-open range. It includes the start value and excludes the end value. So 0..5 produces 0, 1, 2, 3, and 4. This is similar to Python's range(5). If we wanted to include the end value in Rust, we would write 0..=max_steps, but that would be wrong here because max_steps = 5 should mean five attempts, not six.

The Final arm returns early:

StepOutcome::Final(answer) => return Ok(answer),

That return exits the whole run_loop function, not only the match.

The ToolExecuted arm does nothing:

StepOutcome::ToolExecuted { .. } => {}

That empty block simply lets the loop continue to the next iteration.

If every allowed step is used and no final answer appears, the function returns:

Err(LoopError::MaxStepsExceeded { max_steps })

max_steps is the loop's backstop. It prevents a model from calling tools forever. This is not a sophisticated policy yet, but it is a necessary boundary.

8.7 MaxStepsExceeded

Chapter 7's LoopError represented provider failures, parse failures, unknown tools, and tool failures. Chapter 8 adds one more variant:

File: abcb/crates/abcb-core/src/lib.rs

pub enum LoopError {
    Provider(ProviderError),
    Parse(ModelOutputError),
    UnknownTool(String),
    Tool(ToolError),
    MaxStepsExceeded { max_steps: usize },
}

This error belongs to the loop layer. No provider produced it. No tool produced it. The loop produced it because the model did not reach a final answer within the allowed number of steps.

Is reaching max_steps really an error?

In this API, yes. The caller asked the loop for a final answer, and the loop could not produce one within the configured budget. That does not mean the model or tool crashed. It means the loop hit its safety boundary. We represent that as LoopError::MaxStepsExceeded because the caller needs to handle it differently from a successful final answer.

The display message says that directly:

File: abcb/crates/abcb-core/src/lib.rs

LoopError::MaxStepsExceeded { max_steps } => {
    write!(f, "exceeded max steps ({max_steps}) without a final answer")
}

The test forces the case:

File: abcb/crates/abcb-core/src/lib.rs

#[test]
fn run_loop_errors_when_max_steps_exceeded() {
    let mut provider = MockProvider::new([
        r#"{"kind":"tool_call","tool_name":"stub_echo","arguments":{"text":"a"}}"#,
        r#"{"kind":"tool_call","tool_name":"stub_echo","arguments":{"text":"b"}}"#,
    ]);
    let registry = registry_with_stub_echo();

    let err = run_loop(&mut provider, &registry, "hi", 2).expect_err("loop should fail");

    assert!(matches!(err, LoopError::MaxStepsExceeded { max_steps: 2 }));
}

The provider gives two tool calls and never gives a final answer. With max_steps set to 2, the loop executes both steps and then stops with MaxStepsExceeded.

8.8 The Tests That Define the Loop

The happy path can finish immediately:

File: abcb/crates/abcb-core/src/lib.rs

#[test]
fn run_loop_returns_final_immediately() {
    let mut provider = MockProvider::new([r#"{"kind":"final","content":"done"}"#]);
    let registry = registry_with_stub_echo();

    let answer = run_loop(&mut provider, &registry, "hi", 5).expect("loop should succeed");

    assert_eq!(answer, "done");
}

Or it can run a tool and then finish:

File: abcb/crates/abcb-core/src/lib.rs

#[test]
fn run_loop_executes_tool_then_returns_final() {
    let mut provider = MockProvider::new([
        r#"{"kind":"tool_call","tool_name":"stub_echo","arguments":{"text":"pong"}}"#,
        r#"{"kind":"final","content":"finished"}"#,
    ]);
    let registry = registry_with_stub_echo();

    let answer = run_loop(&mut provider, &registry, "hi", 5).expect("loop should succeed");

    assert_eq!(answer, "finished");
}

Or it can run multiple tools before the final answer:

File: abcb/crates/abcb-core/src/lib.rs

#[test]
fn run_loop_runs_multiple_tools_then_final() {
    let mut provider = MockProvider::new([
        r#"{"kind":"tool_call","tool_name":"stub_echo","arguments":{"text":"one"}}"#,
        r#"{"kind":"tool_call","tool_name":"stub_echo","arguments":{"text":"two"}}"#,
        r#"{"kind":"final","content":"both done"}"#,
    ]);
    let registry = registry_with_stub_echo();

    let answer = run_loop(&mut provider, &registry, "hi", 5).expect("loop should succeed");

    assert_eq!(answer, "both done");
}

The mock provider is simple, but that simplicity is useful. We can script the model's decisions and verify how the loop responds.

In a real model call, the second response would depend on the tool result that was appended to the session. The mock provider does not reason about that session. It only returns the next scripted string from its queue:

File: abcb/crates/abcb-core/src/lib.rs

let mut provider = MockProvider::new([
    r#"{"kind":"tool_call","tool_name":"stub_echo","arguments":{"text":"pong"}}"#,
    r#"{"kind":"final","content":"finished"}"#,
]);

That means the test proves the loop control flow, not model reasoning. The session mutation test from 8.4 proves the mechanism that a real provider would rely on: the tool output is actually appended to the session before the next provider call.

8.9 Who Owns the Session?

For now, run_loop creates the session internally:

File: abcb/crates/abcb-core/src/lib.rs

let mut session = Session::new("run-loop");
session.push_message(Message::new(Role::User, user_message));

That is convenient for this chapter. The caller gives a user message, and run_loop handles the rest.

But this ownership choice will not be final forever.

Eventually, the CLI will need more control over the session. It may need a real session id before the loop starts. It may need to choose where session files live. It may need to load a previous session from disk. When that happens, session ownership will move outward, and the caller will pass a session into the loop.

This is one of those design choices that is right for now, but not permanent. That is not a failure. It is part of building the framework step by step. At this point in the book, keeping run_loop simple is more valuable than designing the final session lifecycle too early.

The important thing is that run_step already has the more flexible shape:

pub fn run_step(
    provider: &mut impl Provider,
    registry: &ToolRegistry,
    session: &mut Session,
) -> Result<StepOutcome, LoopError>

run_step does not care who owns the session. It only needs a mutable borrow. So when session ownership moves outward later, the step function will already fit.

8.10 The CLI Uses the Loop

The CLI's run command now calls run_loop instead of run_step:

File: abcb/crates/abcb-cli/src/main.rs

/// Default ceiling on agent-loop iterations until config wiring lands (P3-T01).
const DEFAULT_MAX_STEPS: usize = 5;

File: abcb/crates/abcb-cli/src/main.rs

fn run_run(message: String, mock: bool) -> Result<(), Box<dyn Error>> {
    if !mock {
        return Err(
            "only --mock is supported right now; pass --mock to use the mock provider".into(),
        );
    }

    let mut provider = MockProvider::new([format!(
        r#"{{"kind":"final","content":"mock run: you said {message}"}}"#
    )]);
    let registry = default_registry(PathBuf::from(NOTES_PATH));

    let answer = run_loop(&mut provider, &registry, &message, DEFAULT_MAX_STEPS)?;
    println!("{answer}");

    Ok(())
}

The mock CLI path still returns a final answer immediately. That keeps the command usable while no real provider exists.

The important change is not the mock response. It is that the CLI now goes through the same core run_loop function as the tests. The command is wired to the agent loop, even if the default mock behavior does not yet exercise tool calls from the terminal.

8.11 What We Chose

This chapter makes the first real agent loop.

The loop owns a session, adds the user's message, and repeatedly calls run_step. Each step can return a final answer or execute a tool. Tool results are appended as Role::Tool messages so the next provider call can see them.

Rust's ownership model shows up directly in that design. The loop owns the session. The step borrows it mutably. The mutable borrow lets history accumulate without moving the session in and out of each call.

max_steps is the loop's only termination backstop for now. If the model never produces a final answer, the loop stops with LoopError::MaxStepsExceeded.

There is still no recovery policy. If parsing fails, if the model asks for an unknown tool, or if a tool rejects its arguments, the ? in run_loop returns the error immediately. That is the next chapter. Chapter 8 makes the loop repeat. Chapter 9 will teach it how to fail more usefully.

To be continued