Goodbye Slop; Welcome Determinism

Hello everyone. So, goodbye slop, welcome determinism. I know this seems contrary to a few of the talks here, but I'm going to show you there's actually a lot of overlap here. Our goal at agent Conf, we want to make agents that are predictable and we really want to get rid of the slop because our users don't really want to see that. So, who am I? Just real quick, I'm @DavidKPiano pretty much everywhere. That's my handle. My name is David Khourshid. I created a state machine library called XState. How many of you have used or know what XState is? All right, thank you for the lights because I'm blind the PR. I can't see your hands, but yeah, a good amount of you. And I am also the founder of a company called stately.ai. I'm not trying to sell you anything. You could do this with pretty much anything.

XState's an open source library. And also piano's not my last name. I'm a pianist just like this guy. You might know him. Chopin, I hope you do. We're in Poland. All right, so first of all, some, you know, groundwork. What is a state machine? Now, I could give you the mathematical definition or I could show you a visual diagram, which of course I'm going to do, but instead I want to point you to this very simple relation. Given the state and given an event, you are expected to reach a next state. This is how pretty much every single application that you've ever built works. You're in the state, some event happens and predictably you're in some next state. And this is what we call deterministic and this is going to be a key point in this entire talk. So, I recently well, I didn't have the baby but I am a father to a baby.

It's like my new thing right now and [laughter] and he is 6 months old right now but when he was born, it was actually embarrassingly deterministic. He's crying, you feed him, he's eating, he's fed and now he's sleeping and then he wakes up and then you repeat this process every half hour every hour, you know, you you have no sleep. But basically this is the simple infinite loop. And then at around 3 months it got a little bit crazier. So this state machine got a lot more complicated and right now it is it's even worse than this. There's some mysterious discontent. I had trouble figuring that out. Right now we're going through sleep regression. So like nothing works. You just have to pray for a miracle and then they'll go to sleep.

Play from 0:00

But yeah, so babies pretty much went from loops to graphs for me. Just like our whole hype cycle of agents. You saw everyone talking about loops and now people are talking about graphs. They're the same picture. They're the same thing actually. Loops are graphs and so what I'm going to be doing is doubling down on this whole theme here about using graphs in agent engineering. So funny enough, I actually talked about this or a related topic at the previous agent conf which was 3 years ago. And back then it was more about constraining agent behaviors to state machines. So today I'm actually going to be talking about how we could use state machines and determinism to improve agent behavior and to remove slop. So where does slop in agents come from? In my opinion, it comes from unstructured delegation. And here's what I mean by this. When you tell an agent to basically do everything, you let it do judgments, you give all the structure over to an agent, you let it do all the implementation and taste.

Agents and models are getting smarter and more powerful, and it could actually do a lot more things now where it's not hallucinating or doing silly things. However, there's still always the risk of slop when you're just throwing things over the wall and you're hoping for the best from the agent. So, I'm not arguing that we throw this away and that we say, "no, agent, you must do these specific things." I'm going to demonstrate the opposite soon. And of course, we say it works, so who cares? The problem is it works until it doesn't. And when it stops working, what can we actually point to tell the agent, "this is what you need to fix."? So, one of the key things to fixing this is determinism. Where, in simple terms, determinism means given the same input, you get the same predictable output every single time.

Play from 3:00

Again, this seems opposite to what agents and AI are, but I promise you it is actually going to be useful specifically for how we improve agent behavior. So, I'm going to do this because I only have around 15 minutes. We're going to be doing this with a simple example. And the example is it's an email drafter. So, you say, "I want to draft an email about this." And then this agent that we're building is actually going to help you draft an email. So, again, very simple, and it is a distillation of some of the most complex problems that I've been working on for the last few months, but I wanted a simple example to explain it. So, where does determinism fit in here? Because obviously, emails, there could be infinite possibilities for them, but determinism lives in the rules. For example, you must draft an email before you send it. You can only send approved emails.

So, emails that the user says, "yes, I approve. This is the email I want to send." And you must never invent a recipient, obviously. The non-deterministic parts are, for example, the email content, the questions that the agent may ask you to clarify what you want to include in the email, and of course, the drafts made when it is going through that feedback loop and iterating over your requests. So, you might think, "you know what? I know how to do this. I'm just going to write a mega prompt. I'm going to have some rules over here. I'm going to give it a tool call to send the email." This works, of course, but there is a problem with this. First of all, we're encoding control flow inside the prompt. We're saying, "first do this, then do that. If the user asks for changes, revise the draft, and never call send email until the user explicitly approve the email." And we're also repeating that in the tool call, too.

So, in the description, we're saying, "only call this after the user approves it." Obviously, you should never write tools like this where the tool is part of the control flow, but if you're doing a mega prompt and you're trying to have the agent do everything, that's the problem you have. And also, when you don't split up the steps, you're you're pretty much restricted to a single model, a single, you know, system prompt, single settings, instead of having variety in each of the steps that happen. So, of course, this works most of the time, but the problem is when you have this black box of a prompt, you can't prove that it works. You can't even see what the agent is thinking or how it's working, even if you turn reasoning on, etc. You can't guarantee that it's going to work every single time. And the rules decay with context because that prompt, the rules in there, are just it's text just like the rest of your context.

So, as your context builds up, the rules get a little bit less significant over time. And like I said, with one prompt, you're limited to one strategy instead of having different strategies for different parts of your agentic workflow. So, what would this look like with a more deterministic system? For example, if we create a state machine, this is what that state machine may look like where, you know, we initially make the request to drafting, and then we go into the reviewing state, and in the reviewing state we could say, you know, we need clarification or we need to make modifications, things like that. And we also have some safeguards. For example, we should never send an email without a recipient. And so, when we provide a recipient, then we're sending. And then, once we decide to send it, only then can the email be sent. So, in this, it's actually mathematically impossible to send an email when you're in one of in the drafting state or the reviewing state because there just doesn't exist any transition there. So, let's do a quick demo and see how this might look.

All right, here we go. So, here is an example of an email drafter, and this is being automatically visualized from a state machine that I actually an agent wrote this, but still works. So, I'm going to say, "email Jenny, invite them for coffee after my talk." It's intentionally vague. It's saying it needs more info, and it says, "Hey, there are some details missing. What's her email address, etc.?" And I'm just going or what is the subject?" Subject is coffee. And so, it's going to add that information. It's still going to ask for more info, like what should the body say? I'm going to say draft anyway just because I don't have too much time. It's reviewing. Oh, wait, it says best your name. Okay, my name is David. And so, you can see it's actually going through each of the steps. And then we're we're just sending the email. But, of course, we forgot the email address.

And you might be looking at this and saying, "yeah, this is pretty cool, but it's not the best workflow." And that is exactly the point that I'm going to show next. So then finally, we have the email sent to a simulated outbox. But, that is that's an agent working from a state machine. Now, one of the things you might be asking yourself is, "isn't this too much ceremony?" And in my opinion, I would say that modeling is not ceremony when it actually replaces confusion, both for you and for the agents. It's actually extremely useful to basically have a common understanding of how your agent behaves, for both your agents to introspect and for you to understand each of the steps. And of course, not everything needs to be formally modeled as a state machine. I'm not saying that you should do this with every single one of your agents, but where it applies, it's really, really useful. And with this, you also have observability. You have logs where it's not just plain text, it's not just agent thinking or something.

You have structured logs for what states you were in, what events happened, and like this is a simpler example, this is a more complex example. But, at the end of the day, both of them you could see have reached the same state, which is sent, and that we have sent in email. So, I'm not arguing that explicit, like just modeling everything as a state machine or a workflow, is good. And this is this is the next point. I'm saying that when things are explicit, and you have an explicit structure like a state machine, it's inspectable. And in my opinion, that's a lot more useful than yeah, just having an implicit structure. Because you cannot draw arrows in paragraphs. You can't say like, you know, in this prompt, this is where it failed, or this is where we're having problems, or this is where we could adjust things. It's all just text.

And so, with a state machine, you actually have that ability. And so, this is what got me thinking and experimenting a few months ago, where when we have a state machine workflow representing agent behavior, we could do evals on each of the steps, of course. So, we could say when we're drafting an email, this is the expected output from this input. Or we could evaluate the entire outcome of the workflow, where we could say, given this initial set of input, assuming that we go through the whole process, the overall output should be, you know, as we expected, and we could use LLM as judge, etc. But one thing we're missing is actually evalling the edges. So, when you saw the previous demo, you were probably thinking like, as a user, okay, this works, it does get me to the end state of having an email, but it's a little bit annoying. So, this is where you could actually take that state machine as a durable artifact, and you could pass it to an agent and say, here's my state machine, here's all the traces from a few of the sample workflows that we've done through this with.

It could be real users, it could be simulated users, like you dog fooding your agent. And here's actually all of the evals from each of the steps and the overall evals from machine. Please propose some changes to this agent. And I've done this with software factories at Stately. We've done it with our state generation or state machine generation and editing pipeline. And so this actually works really well because you're essentially saying, "Hey, here's a state machine that you're working from. Look at it and see how you could actually improve it yourself." And so keep doing the wrong key command. I actually did that and here we go. So, what it actually produced was a V2 over here. And if you find me after the talk, I could show you a diff of it and you know, go into more detail. But now this state machine changed. It found improvements. It saw that we were doing multiple passes just to fix small issues in the email when instead we could actually draft the email first and then figure out what's needed after.

So, it goes straight to reviewing. It drafted an email. I could make changes like my name is David. And it could add that, go to reviewing, and then it could send the email. And this is actually a much quicker workflow. And then you could actually compare this against what you did before and you could see like, "Hey, this actually feels better." So, all of your evals could pass with the first machine, but unless you're dog fooding and you're going through these iteration processes where you're actually having the agent improve its own state machine, you know, you're not going to get the optimal results until you actually do that. And with state machines you actually have something where you could do that. So, I would argue that to implement this, again, you don't need a special library, you don't need a special tool. In fact, this is pretty much the only loop you need, where you have some sort of initial state, and so while the state is active, or whatever your definition of not done is, you're executing, for example, LLM calls, tool calls, async processes that might be outside of the agent, and you're determining what is the next state and the effects to execute based on the current state and the events that just happened.

And so, that machine, that logic, it could be XState, you could write a switch statement, a reducer function. Coding agents are really good at updating these things, regardless of whatever you use. You could use LangGraph, I guess, which actually is, one of the, most popular tools for using state machines with agents. And updating an agent in an agent's logic follows this loop, and this is what I talked about previously, but you create the model, so you create the state machine, then you observe it, so, given the state machine, you're able to observe every one of the transitions that happened, and from that, you could actually have the agent propose, not the exact same agent, but a different agent that's actually introspecting and looking at its own state machine and saying, "I have ideas for how we could use our current workflow and use the traces to actually improve this workflow." And then you could compare the two and actually run pretty much the same, scenarios on machine version one and machine version two, and you could see, "did this actually make an improvement?" And there's a lot of prior arts over here, agent spec, A flow, and also even older than that, about 20 years older, process mining as well, where it's essentially the same thing.

We're going over traces of events, and we're distilling that into some sort of formal model, which is a state machine, and using that we're identifying where we could optimize. So, I'm not saying that you could use a state machine and say, "this is the way that the agent should always run." Instead, I'm saying the state machine is how you could have the agent self-improve. So, again, in 2023, at the last agent Conf, I was talking about how the graph keeps the agent honest. It gives it guardrails, it gives it boundaries to work from. And in 2026, I'm talking about how using that exact same graph thinking can make the agent improve on its own behavior. And obviously, I'm excited about graphs and state machines. A couple days ago, I was pretty much nerd sniped when type safe announced Jeb. How many of you have heard of Jeb or played around with it? No? So, of course, I got excited about this, and but I was busy working on my talk.

But last night, at like 1:00 a.m., because I couldn't sleep because the baby was awake, I was like, "you know what? I really want to try making a demo of this." So, in 2023, I was showing how an espresso machine, which is a state machine, can be used with an agent. In an agent, you could give it drink orders, and it would know exactly how to operate the machine to reach that state where it, produced a drink. And I was, yeah, just to show I was nerd sniped, I said, "this is going to pair really well with state machines." And the creator said, "you're absolutely correct." So, you know what?

Play from 5:00

Let's let's just try this real quick. I have like a minute. This is all right, here's my updated espresso machine and this is using Jev. So, basically, this is a giant state machine under the hood and Jev is given the exact state machine and also all of the possible events that can that the virtual barista can do. So, we could say I want a cap and also a shot of I'm going to spell this wrong on purpose. Ex- expresso. And so, we press order and it's actually doing the order and it's hopefully you could see that. It it's doing each of the steps and also I introduced chaos here so sometimes parts of the machine would break. So, let's say also a cord- tado. Definitely getting one after this talk. And so, you could see that this is real time. This is Jev actually doing every single one of the steps for making each one of these drinks.

It's it's not pre-programmed. Jev just knows like, "Hey, given it it's showing the likelihood over here." So, you know, it's saying, "Okay, we probably want to do a steam wand." let's let's break some stuff. So, let me break the grinder. So, Jev's going to decide while it's working, I should probably fix the grinder because you know, obviously you can't grind beans and make espresso drinks unless the grinder fixed. And so, you could see it's it's all working. I'm actually surprised. All right. Thank you.

Play from 19:00