See Part 0 for the latest table of contents and sample code. New chapters will be added over time.

Chapter 4: The Provider Trait and a Scripted Mock

The previous chapter gave abcb a conversation shape. We now have Role, Message, and Session, and those types can leave memory as JSON and come back intact.

This post is also available on Medium. If you’re a paid Medium member and happen to read it there, it helps fund my next cup of coffee. Much appreciated ☕️😄

But a conversation by itself does not produce a reply. Something has to take the current session, ask a model for the next response, and bring the assistant message back into the framework.

That something is the provider.

In this series, a provider is the part of the framework that knows how to ask for the next response. Later, that will mean an HTTP provider that talks to a local OpenAI-compatible model server. But if we begin there, too many things arrive at once: network calls, async code, model configuration, server errors, and the unpredictable behavior of the model itself. We need something smaller first.

So this chapter adds two things:

• a Provider trait, the contract for anything that can produce the next assistant message
• a MockProvider, a scripted in-process implementation of that trait that returns canned replies

This is not a toy detour. A mock provider is how we make the framework testable before the real model arrives. Rust can use mock libraries, but for a small trait like this, a hand-written implementation is often simpler and clearer. It also lets us start building the agent loop without spending tokens, waiting on a server, or wondering whether a strange answer was caused by the framework or by the model.

The sample code for this chapter is in chapter04/abcb/.

4.1 What a Provider Is

The new provider boundary is written as a Rust trait. A trait is close to what many languages call an interface: it describes behavior that some type must provide. We will look at traits more directly in the next section. For now, the important part is that Provider names what the framework needs, without choosing one specific implementation yet.

The trait itself is short:

File: abcb/crates/abcb-core/src/lib.rs

pub trait Provider {
    fn complete(&mut self, session: &Session) -> Result<Message, ProviderError>;
}

Read the signature as a sentence:

Given mutable access to a provider and borrowed access to a session, try to produce a message. If that fails, return a provider error.

That sentence is the first real abstraction boundary in the framework.

The provider does not own the session. It receives &Session, a shared reference. That means it can inspect the conversation, but it cannot change it. This is important because the session is the framework's conversation state. A provider should not secretly mutate the transcript while producing a reply.

The provider does, however, receive &mut self. That means the provider itself may change while producing a reply. The scripted mock needs this because each call removes one canned response from its internal list and returns it as the assistant's message. A future HTTP provider may use mutable state for counters, cached configuration, retry bookkeeping, or connection-related state. We do not need all of that yet, but the signature leaves room for stateful providers.

The return type is Result<Message, ProviderError>. A provider may produce the next assistant message, or it may fail. The trait does not hide that possibility. It puts failure into the type signature. One interesting part is that we define a provider-specific error type instead of using a plain string or panicking. We will look at that next.

4.2 Traits

A trait in Rust describes behavior that a type can provide. In other languages, the closest familiar word may be "interface," but Rust traits also participate in generics, implementations, bounds, and dispatch choices in ways that become important later.

For now, the useful idea is simple: Provider says what behavior the framework needs, without saying which concrete provider will do it.

File: abcb/crates/abcb-core/src/lib.rs

pub trait Provider {
    fn complete(&mut self, session: &Session) -> Result<Message, ProviderError>;
}

The trait only names the method. It does not include a body. Any type that wants to be a provider must implement this method.

That lets the rest of the framework depend on the concept of a provider instead of depending on one concrete implementation. Today the implementation is MockProvider. Later it can be an HTTP provider. The code that calls complete should not need to know which one it is using.

This is the first place where Rust's type system starts to look like architecture. The trait is not only a language feature. It is a boundary and a contract.

4.3 Errors Are Part of the Contract

The provider can fail, so the chapter adds a small error type.

File: abcb/crates/abcb-core/src/lib.rs

#[derive(Debug, Eq, PartialEq)]
pub enum ProviderError {
    NoMoreResponses,
}

Right now there is only one error. The mock provider can run out of scripted responses. That is not a panic. It is a normal failure case, and the caller should be able to test for it.

The enum is intentionally small. Later provider errors will grow: network failures, invalid responses, model server errors, maybe configuration errors. But this chapter only needs the one real error the mock can produce.

The type also implements Display and the standard Error trait:

File: abcb/crates/abcb-core/src/lib.rs

impl fmt::Display for ProviderError {
    fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
        match self {
            ProviderError::NoMoreResponses => write!(f, "no more scripted responses"),
        }
    }
}

impl std::error::Error for ProviderError {}

Display is for human-readable text. The CLI can turn the error into a message. Test output also becomes easier to read.

Notice that ProviderError did not have to declare all of its supported traits inside the enum definition. In Rust, a type supports a trait when there is an impl TraitName for TypeName block. We defined the enum first, then attached Display behavior with impl fmt::Display for ProviderError.

std::error::Error marks the type as an error type in Rust's standard error ecosystem. The implementation is empty because the default behavior is enough for now. The important part is that ProviderError can now behave like a normal Rust error.

The path std::error::Error is also worth reading slowly. std is Rust's standard library crate. error is a module inside that crate. Error is the trait inside that module. The :: syntax walks through that path from crate, to module, to item. We saw the same shape with fmt::Display: fmt is the module path we imported, and Display is the trait inside it.

There is also a small lifetime marker in fmt::Formatter<'_>. Chapter 3 introduced named lifetimes like 'a. Formatter itself is a type with a lifetime parameter. Conceptually, it has this shape:

pub struct Formatter<'a> {
    // fields omitted
}

In the Display implementation above, fmt::Formatter<'_> uses '_, the anonymous lifetime, sometimes also called the placeholder lifetime. It means "there is a lifetime parameter here, but let the compiler infer its exact name." We could give it a name if the relationship needed to be reused, but this signature does not need one. We could also elide the lifetime completely in this position and write fmt::Formatter, but then the lifetime parameter would be hidden from the reader.

Elide and elision

To elide something means to leave it out. Elision is the act of leaving it out. In Rust, lifetime elision means Rust lets us omit a lifetime annotation when the compiler can infer it from the surrounding code. If we wrote f: &mut fmt::Formatter without <'_>, the lifetime parameter would be elided. Rust can usually still infer the hidden lifetime in this position.

But the lifetime is still there, and some projects choose to warn or fail when lifetimes in type paths are hidden. Writing Formatter<'_> is a middle ground: the lifetime slot is visible, but we do not give it a reusable name.

This is different from the cases in Chapter 3 where a named lifetime was needed to describe a relationship, such as BorrowedMessage<'a> or fn longest<'a>(x: &'a str, y: &'a str) -> &'a str. In those cases, omitting the lifetime is not just a style choice. The compiler needs the relationship to be written down. Here, the formatter is borrowed only for the duration of the formatting call, so the placeholder lifetime is enough.

4.4 The Scripted Mock

The mock provider stores a queue of responses.

File: abcb/crates/abcb-core/src/lib.rs

pub struct MockProvider {
    scripted: VecDeque<String>,
}

VecDeque is a double-ended queue from Rust's standard library. We use it because the mock consumes responses from the front. A plain Vec could work, but repeatedly removing from the front of a vector is not its best shape. VecDeque makes "push at the back, pop from the front" feel natural.

The mock is scripted, which is why the struct field is named scripted. It is not constant or echo-based. A constant mock would return the same reply forever. An echo mock would only mirror the user's message. A scripted mock lets tests say, "first return this, then return that, then fail." That will matter more once the loop has multiple turns.

The constructor accepts anything iterable whose items can become strings.

File: abcb/crates/abcb-core/src/lib.rs

impl MockProvider {
    pub fn new<I, S>(responses: I) -> Self
    where
        I: IntoIterator<Item = S>,
        S: Into<String>,
    {
        Self {
            scripted: responses.into_iter().map(Into::into).collect(),
        }
    }
}

This looks more complicated than the previous constructors, so it is worth reading slowly.

MockProvider::new is an associated function, like the constructors we saw before. It does not take self because there is no provider yet. It builds one.

I and S are generic type parameters. In this function, I is the type of the responses argument. S is the type of each item produced when responses is iterated. The caller does not write these types explicitly in normal use. Rust infers them from the value passed to MockProvider::new.

I: IntoIterator<Item = S> is a trait bound. It says the type I must implement the IntoIterator trait, and when it is turned into an iterator, that iterator must yield items of type S. In everyday speech, we might say "I implements IntoIterator." That lets callers pass arrays, vectors, or other iterable collections.

S: Into<String> is another trait bound. It says the item type S must implement the Into<String> trait, which means each item can be converted into an owned String. That lets callers pass string literals like "first" or owned strings.

Trait bounds

A bare generic type can do almost nothing by itself. If a function receives a T, Rust does not know whether T can be cloned, compared, printed, iterated, or converted. Bounds add those capabilities to the type parameter.

fn is_larger<T: PartialOrd>(left: &T, right: &T) -> bool {
    left > right
}

Multiple bounds can be combined with +:

fn show<T: Clone + std::fmt::Debug>(x: T) {
    let copied = x.clone();
    println!("{copied:?}");
}

When bounds get long, Rust lets us move them into a where clause, as MockProvider::new does. The meaning is the same, but the signature is easier to read.

4.4.1 The Iterator Chain

Inside the function, this line does the actual conversion:

scripted: responses.into_iter().map(Into::into).collect(),

Rust iterator chains often read from left to right. Here, responses.into_iter() turns the input into an iterator. Because this is into_iter, it takes ownership of the input and yields owned items from it. That is fine for the constructor because we do not need the original responses value after building the provider.

The three common ways to start iterating over a collection are worth separating. Suppose responses is a collection whose items have type T:

That distinction matters because the next method, map, receives whatever the iterator yields. If an iterator yields &i32, the mapping function receives &i32. If it yields String, the mapping function receives String.

For example, this copies integers out of a vector before summing them:

let v: Vec<i32> = vec![1, 2, 3];
let sum: i32 = v.iter().copied().sum();

Without .copied(), v.iter() yields &i32. In this exact sum example, Rust can still sum references to integers because the standard library supports it. But the iterator is still yielding references. For small copyable values like i32, .copied() makes that explicit by turning each &i32 into an i32.

In our constructor chain, responses.into_iter().map(Into::into).collect(), the middle step converts each yielded item into a String. Into::into is the function form of calling .into() on each item. It is a compact way to write this:

scripted: responses
    .into_iter()
    .map(|response| response.into())
    .collect(),

The last method, .collect(), gathers the iterator back into a collection. Without collect, the chain would still be an iterator value, not the VecDeque<String> that the scripted field needs. Rust infers that the target collection is VecDeque<String> because the value is assigned to the scripted field, and that field has type VecDeque<String>.

This is a common Rust shape: start from a collection, turn it into an iterator, apply iterator adapters such as map, filter, enumerate, or zip, then finish with a consumer such as collect, sum, or a for loop. The adapter methods live on iterators, not directly on every collection type. That is why we write responses.into_iter().map(...) instead of responses.map(...).

Once this shape becomes familiar, many Rust loops become small iterator pipelines. Here are a few examples:

let doubled_positive: Vec<i32> = values
    .iter()
    .filter(|&&x| x > 0)
    .map(|&x| x * 2)
    .collect();

let sum_of_squares: i32 = values.iter().map(|&x| x * x).sum();

for (index, &value) in values.iter().enumerate() {
    println!("{index}: {value}");
}

for (&left, &right) in left_values.iter().zip(right_values.iter()) {
    println!("{left} + {right}");
}

These methods do not change the idea of ownership. They make it visible. iter borrows, iter_mut mutably borrows, and into_iter consumes. The rest of the chain works with the kind of item that first choice produces.

As we saw in the trait bounds note, our constructor could be written without a where clause like this:

pub fn new<I: IntoIterator<Item = S>, S: Into<String>>(responses: I) -> Self

That is the same idea, but the constraints sit inside the angle brackets. The where form separates the type names from the requirements on those types. Once a signature has more than one bound, I usually find the where form easier to read.

We have already used this idea in a smaller form. Message::user(content: impl Into<String>) said, "accept anything that can turn into a String." The mock constructor is the same pattern with two generic pieces instead of one.

One more detail hides inside IntoIterator<Item = S>. Item is an associated type of the IntoIterator trait. An associated type is a type member declared by a trait and chosen by each implementation of that trait.

Conceptually, a trait can say something like this:

trait Iterator {
    type Item;

    fn next(&mut self) -> Option<Self::Item>;
}

Each implementation chooses what Item means for that implementing type:

impl Iterator for Counter {
    type Item = u32;
}

In that impl Iterator for Counter block, Self means the implementing type. So Self is Counter, and Self::Item is u32.

That is why the bound is written as I: IntoIterator<Item = S>. We are saying: "I must implement IntoIterator, and the associated Item type of that implementation must be S." It is not Item: S, because S is not a trait to implement. It is the concrete item type we want the iterator to yield.

The result is a flexible constructor with a simple internal shape. The constructor is flexible because callers can pass different kinds of input. Arrays and vectors implement IntoIterator, and string literals and String values can become owned strings through Into<String>. But after construction, the mock always stores the same thing: a VecDeque<String> of owned responses. The caller gets convenience at the boundary. The provider keeps ownership inside.

4.4.2 Reading Rust Signatures

This constructor also gives us a good excuse to pause on Rust signature syntax. Rust function signatures can look crowded because they can mention several different things at once: parameter names, parameter types, lifetimes, generic type parameters, trait bounds, return types, and sometimes const generics.

The rough shape is:

fn name<generic_parameters>(parameter_name: ParameterType) -> ReturnType
where
    bounds,
{
    // body
}

The generic parameter list goes after the function name:

fn example<'a, T, const N: usize>(value: &'a T, items: [T; N])
where
    T: Clone,
{
}

That list can contain lifetime parameters like 'a, type parameters like T, and const generics like const N: usize. If we want to use a named lifetime such as 'a in the parameter or return types, we first introduce that name in the generic parameter list. Without <'a>, a signature cannot suddenly use &'a T. The alternative is to let Rust elide the lifetime, or to use the placeholder lifetime '_ when a lifetime slot should stay visible but does not need a reusable name. Const generics let a value, often an array length, participate in a type. We do not need them for MockProvider, but they are part of the same signature grammar.

A function parameter is written as name: Type:

fn send(message: Message) {}
fn send_ref(message: &Message) {}
fn send_mut(message: &mut Message) {}

References put & or &mut in the type. If a lifetime must be written, it goes after the &:

fn borrowed<'a>(text: &'a str) {}
fn borrowed_mut<'a>(text: &'a mut String) {}

There are two related syntaxes here. In an expression, &value borrows a value. In a type, &Type means "a reference to Type." Function signatures use the type form after the colon: text: &str, or with an explicit lifetime, text: &'a str. The lifetime belongs to the reference, so it sits between & and the referenced type.

Trait bounds can appear in two common places. For a named generic type parameter, the bound can go in the generic parameter list:

fn show<T: std::fmt::Debug>(value: T) {}

or in a where clause:

fn show<T>(value: T)
where
    T: std::fmt::Debug,
{
}

For function parameters, Rust also has impl Trait syntax:

fn user(content: impl Into<String>) {}

Read this as "accept some type that implements Into<String>." It is useful when the caller does not need to know the type parameter's name. The named-generic version is:

fn user<T: Into<String>>(content: T) {}

What we do not write is T: impl Into<String>. If the type parameter is named T, the bound is written as T: Into<String>. If we do not want to name the type parameter, we can write the parameter type as impl Into<String>.

There is also dyn Trait, which means a trait object. It is used when the concrete type is chosen at runtime through a pointer-like value:

fn log(error: &dyn std::error::Error) {}

That is different from impl Trait, which still means one concrete type chosen by the compiler for that call or return position. We will not need trait objects yet, but the notation is common enough that it is worth recognizing.

'static can appear where a lifetime is expected:

let text: &'static str = "hello";

or as a bound:

fn keep<T: 'static>(value: T) {}

In the first case, the reference is valid for the whole program. In the second case, T: 'static means the type does not contain non-static borrowed data. We will return to this when the framework starts storing longer-lived values.

The same grammar appears outside function signatures too. Structs, enums, traits, and impl blocks can all carry generic parameters and bounds.

For a struct, the generic parameter list goes after the struct name:

struct Holder<T> {
    value: T,
}

struct Borrowed<'a> {
    text: &'a str,
}

Enums can do the same:

enum Maybe<T> {
    Some(T),
    None,
}

Traits can also have generic parameters, associated types, and method signatures:

trait Store<T> {
    type Error;

    fn save(&mut self, value: T) -> Result<(), Self::Error>;
}

And impl blocks repeat the generic parameters they need:

impl<T> Holder<T> {
    fn new(value: T) -> Self {
        Self { value }
    }
}

The first T in impl<T> introduces the generic name for this implementation block. The second T in Holder<T> applies that generic name to the type being implemented. In other words: "for any T, implement methods on Holder<T>."

Trait implementations can show the same pattern:

impl<T: std::fmt::Debug> Store<T> for DebugStore {
    type Error = std::io::Error;

    fn save(&mut self, value: T) -> Result<(), Self::Error> {
        println!("{value:?}");
        Ok(())
    }
}

Here, impl<T: std::fmt::Debug> introduces the generic type and its bound. Store<T> says which version of the trait is being implemented. DebugStore is the concrete type receiving that implementation.

The pattern is repetitive once you see it. Introduce names in angle brackets, use those names in fields or method signatures, and put bounds either beside the names or in a where clause. MockProvider itself is simple because it stores a concrete VecDeque<String>, but its constructor uses the broader signature grammar to accept many input shapes.

4.5 Implementing the Trait

Now MockProvider implements Provider.

File: abcb/crates/abcb-core/src/lib.rs

impl Provider for MockProvider {
    fn complete(&mut self, _session: &Session) -> Result<Message, ProviderError> {
        let content = self
            .scripted
            .pop_front()
            .ok_or(ProviderError::NoMoreResponses)?;
        Ok(Message::new(Role::Assistant, content))
    }
}

The impl Provider for MockProvider line says: this concrete type satisfies the provider contract.

The method takes &mut self because it mutates the mock. Each call removes one response from the front of the queue. That makes the mock stateful. If a test expects two turns, it can provide two replies. If the framework asks for a third reply, the provider returns an error.

The _session parameter is still present because the trait requires it. The underscore says we intentionally do not use it in this implementation, so Rust should not warn about an unused variable. It is still a normal binding, and using it is not prohibited. If we later used _session inside the function, that would compile. A real provider will read the session. The mock only follows its script.

The most compact line is this one:

File: abcb/crates/abcb-core/src/lib.rs

let content = self
    .scripted
    .pop_front()
    .ok_or(ProviderError::NoMoreResponses)?;

pop_front() returns an Option<String>. If there is a response, it is Some(content). If the queue is empty, it is None.

ok_or(ProviderError::NoMoreResponses) turns that Option into a Result. Some(content) becomes Ok(content). None becomes Err(ProviderError::NoMoreResponses).

The ? operator then says: if this is an error, return it from the current function. It does not only exit the current block. If there is no error, it unwraps the Ok value and keeps going.

That gives us one honest rule: an exhausted mock is a provider error. It is not a surprise crash, and it is not hidden behind a made-up reply.

4.5.1 A Note on unwrap

The word "unwrap" can be confusing here because Rust also has an .unwrap() method. The ? operator unwraps the success value only after deciding that the error case should be returned to the caller. The .unwrap() method is more blunt: it extracts the inner value, or panics if the value is missing.

On Option<T>:

let some_value: Option<i32> = Some(5);
let n = some_value.unwrap(); // n = 5

let none_value: Option<i32> = None;
let n = none_value.unwrap(); // panic

On Result<T, E>:

let ok: Result<i32, String> = Ok(5);
let n = ok.unwrap(); // n = 5

let err: Result<i32, String> = Err("boom".into());
let n = err.unwrap(); // panic

That panic can be fine in tests, quick experiments, or places where an invariant is truly guaranteed. Even then, .expect("reason") is usually better because the panic message explains what assumption was violated.

In framework code, recoverable errors should usually stay recoverable. That is why this chapter uses ok_or(...)? instead of .unwrap():

let content = self
    .scripted
    .pop_front()
    .ok_or(ProviderError::NoMoreResponses)?;

The ok_or part turns Option<String> into Result<String, ProviderError>. The ? part propagates the error if the mock is exhausted. The caller can then decide what to do.

There are other alternatives when panicking is not the right shape:

The rule of thumb is simple: if we cannot clearly explain why the value must be present, we should not use .unwrap() in framework code.

4.6 One Turn

With the trait and mock in place, abcb-core can define the smallest possible loop step.

File: abcb/crates/abcb-core/src/lib.rs

pub fn one_turn(
    provider: &mut impl Provider,
    user_message: impl Into<String>,
) -> Result<Message, ProviderError> {
    let mut session = Session::new("one-turn");
    session.push_message(Message::new(Role::User, user_message));
    provider.complete(&session)
}

This is not the full agent loop. It does not store the session. It does not append the assistant reply back into the conversation. It does not call tools. It does not record events.

But the shape is already visible:

1. create a session
2. add the user message
3. ask a provider for the assistant reply

The provider argument is &mut impl Provider. This means the function accepts a mutable reference to any concrete type that implements Provider. The caller can pass &mut MockProvider today. Later, the same function shape could accept a real provider.

impl Provider here uses static dispatch. The compiler knows the concrete provider type at compile time and generates code for that type. We will revisit the alternative, trait objects with dyn Provider, when the framework needs to store mixed provider types behind one value. For now, impl Provider keeps the code direct.

The function also takes user_message: impl Into<String>, which follows the pattern from Message::new. Callers can pass a string literal or an owned String, and the framework stores owned text.

The last line has no semicolon:

File: abcb/crates/abcb-core/src/lib.rs

provider.complete(&session)

That is the return value of one_turn. If the provider succeeds, the function returns Ok(Message). If the provider fails, the function returns Err(ProviderError).

4.7 Tests Make the Contract Visible

The tests show what the provider contract means in practice.

File: abcb/crates/abcb-core/src/lib.rs

#[test]
fn mock_provider_returns_scripted_responses_in_order() {
    let mut provider = MockProvider::new(["first", "second"]);
    let session = Session::new("s");

    let first = provider.complete(&session).expect("first response");
    assert_eq!(first, Message::new(Role::Assistant, "first"));

    let second = provider.complete(&session).expect("second response");
    assert_eq!(second, Message::new(Role::Assistant, "second"));
}

This test proves that the mock is stateful. It returns "first" and then "second". The provider changes between calls, which is why complete takes &mut self.

The next test proves that exhaustion is explicit.

File: abcb/crates/abcb-core/src/lib.rs

#[test]
fn mock_provider_errors_when_exhausted() {
    let mut provider = MockProvider::new(["only"]);
    let session = Session::new("s");

    provider.complete(&session).expect("first response");
    let err = provider
        .complete(&session)
        .expect_err("provider should be exhausted");
    assert!(matches!(err, ProviderError::NoMoreResponses));
}

matches! is a Rust macro for checking whether a value matches a pattern. Here it says: the error should be the NoMoreResponses variant. We do not need to destructure anything else.

The final core test checks one_turn.

File: abcb/crates/abcb-core/src/lib.rs

#[test]
fn one_turn_returns_assistant_reply_from_provider() {
    let mut provider = MockProvider::new(["bot reply"]);

    let reply = one_turn(&mut provider, "hi").expect("one_turn should produce a reply");

    assert_eq!(reply, Message::new(Role::Assistant, "bot reply"));
}

This test is small, but it proves the abstraction. one_turn does not know how the provider works. It only knows the provider trait. The mock can stand in for the model because both will satisfy the same contract.

4.7.1 Auto-Borrowing and Function Calls

There is a small Rust convenience hidden in the first two tests. The method signature for complete takes &mut self:

fn complete(&mut self, session: &Session) -> Result<Message, ProviderError>;

But the test calls it like this:

let first = provider.complete(&session).expect("first response");

We do not write (&mut provider).complete(&session). Rust's method-call syntax performs automatic borrowing and dereferencing for the receiver, the value before the dot. Because complete needs &mut self, Rust can borrow provider mutably for that method call.

That convenience does not mean Rust inserts &mut everywhere. one_turn is a free function, meaning an ordinary function that is called by name rather than through method-call syntax on a receiver. It is not a method on provider:

pub fn one_turn(
    provider: &mut impl Provider,
    user_message: impl Into<String>,
) -> Result<Message, ProviderError>

Because its first parameter is &mut impl Provider, the call site must pass a mutable reference:

let reply = one_turn(&mut provider, "hi").expect("one_turn should produce a reply");

If we wrote one_turn(provider, "hi"), Rust would try to pass the provider value itself. That would not match the parameter type. We could design one_turn to take ownership of the provider, but that would be the wrong shape for an agent loop. A provider may have state, and we usually want to keep using the same provider across turns.

4.8 The CLI Gets a Mock Chat

The sample also adds a chat subcommand.

File: abcb/crates/abcb-cli/src/main.rs

#[derive(Debug, Subcommand)]
enum Command {
    /// Check the local abcb development environment.
    Doctor,
    /// Send a single user message and print the assistant reply.
    Chat {
        /// The user message to send.
        message: String,
        /// Use the in-process mock provider (required while no real provider exists).
        #[arg(long)]
        mock: bool,
    },
}

The command is deliberately limited. It requires --mock because there is no real provider yet.

File: abcb/crates/abcb-cli/src/main.rs

fn run_chat(message: String, mock: bool) -> Result<(), Box<dyn Error>> {
    if !mock {
        return Err(
            "only --mock is supported right now; pass --mock to use the mock provider".into(),
        );
    }

    let mut provider = MockProvider::new([format!("mock: you said {message}")]);
    let reply = one_turn(&mut provider, &message)?;
    println!("{}", reply.content);

    Ok(())
}

This is not pretending to be an agent yet. It is a command-line smoke test for the shape we just built. The CLI creates a mock provider, sends one user message through one_turn, and prints the assistant message content.

The mock reply uses format!, which creates an owned String. Then MockProvider::new accepts it because each response only needs to implement Into<String>.

The call to one_turn(&mut provider, &message) is also worth noticing. The provider is passed mutably because it may consume a scripted response. The message is passed as a borrowed string because one_turn only needs to turn it into owned content inside the session.

4.8.1 Box<dyn Error>

The return type of run_chat is also doing some work:

fn run_chat(message: String, mock: bool) -> Result<(), Box<dyn Error>>

The success value is (), the unit type, because run_chat does not return useful data. It prints to the terminal and either succeeds or fails.

The error value is Box<dyn Error>. We already touched dyn Trait in the signature section. Here it means: "some value that implements the standard Error trait, chosen at runtime." The concrete error type might be a provider error, a file error, a TOML parsing error, or, in this function, a string converted into an error.

Box puts that error value behind a pointer on the heap. That gives the function one stable return type even when different code paths may produce different concrete error types. Without the box and trait object, the function would need to name one exact error type.

Why not just write dyn Error? Because dyn Error is a trait object, and trait objects do not have a known size by themselves. A function's return type normally needs a known size so Rust knows how much space is needed to move the value around. A trait object says only "some type that implements this trait," so its concrete size is not known from the type alone.

That is why Rust needs to put a trait object behind some kind of pointer, such as &dyn Error, Box<dyn Error>, or another smart pointer. &dyn Error is a borrowed error. It can be useful when the error is owned somewhere else and definitely outlives the borrow, but it is not a good return type for this CLI function. The function may create the error while it is running, so returning a borrowed reference would risk pointing at a value that disappears when the function returns. Box<dyn Error> returns an owned error value behind a pointer, which fits this use case.

Why not write impl Error? In return position, impl Error means "this function returns one concrete error type, but I am hiding its name from the caller." It does not mean "this function may return any error type." If we wanted to avoid Box<dyn Error>, we would usually define one concrete CLI error enum and convert each possible error into that enum. That is often the right choice for library code. For this small CLI edge, Box<dyn Error> is the more flexible fit.

For CLI code, this is a practical tradeoff. The command can use ? with several error sources and let them flow upward as ordinary errors:

let reply = one_turn(&mut provider, &message)?;

The core library is stricter. Provider returns ProviderError, a specific domain error. The CLI is more relaxed because it sits at the edge of the program. It mostly needs to report what went wrong and exit cleanly.

4.9 What We Chose

This chapter makes three decisions that are deliberately time-bound.

First, Provider::complete is synchronous. That keeps the trait simple while the provider is in-process. A real HTTP provider will force us to revisit this. When the model call becomes network I/O, async will earn its place.

Second, complete takes &mut self. This makes the scripted mock easy and honest. It can mutate its queue without RefCell, Mutex, or other interior mutability tools. RefCell lets some borrow checks move from compile time to runtime. Mutex protects shared data across threads. Both are useful, but both add concepts we do not need here. The cost is that callers need mutable access to the provider. For now, that is fine. The framework is still single-threaded and sequential.

Third, the trait lives in abcb-core. We could invent an abcb-models crate or a provider-specific crate now, but that would be a guess. The core crate already owns sessions and messages. The provider trait depends on those types. Keeping it in abcb-core is the simplest honest home until the design asks for a split.

Those decisions may change. That is not a failure. A small framework grows by treating decisions as dated notes: this is what we choose now, this is what it buys us, and this is when we expect to revisit it.

4.10 What We Have So Far

At the end of this chapter, abcb can run one mocked model turn.

It has a Provider trait, a ProviderError, a scripted MockProvider, and a one_turn function. The CLI has a chat --mock command that exercises the same path from the outside.

This still does not call a real model. That is the point. We now have the provider boundary before we have the real provider. The next chapters can build more framework behavior against that boundary without waiting for local HTTP, async, or model-specific response parsing.

The model is still mocked. The shape around it is becoming real.

To be continued