# Model-based testing with state machines

Canonical: https://davidk.dev/talks/model-based-testing-with-state-machines/
Video: https://www.youtube.com/watch?v=LBBC-CiiPQY
Speaker: David Khourshid (@DavidKPiano), creator of XState
Type: Conference talk · Event: Web Zurich meetup · Year: 2020 · Duration: 18:12 · Views: 699 (as of 2026-10-19)
Topics: Model-based testing, State machines, XState

## Summary
Instead of writing your tests, generate them instead! Learn how model-based testing increases test coverage and reliability without having to write tons of tests.

## Key takeaways (quoted from the transcript)
- What I want to talk to you today about is something called model-based testing, and it's a way to write fewer tests. (0:00)
- Once we do that, and once we generate all these tests that go from one state to the final state and go through all of these different paths. (6:00)
- We could basically determine how to get from one state, the initial states to every single other states, which is exactly what we want to test. (10:00)
- Testing library, but we could use Puppeteer as well as just an XState to actually automate these tests. (15:00)

## Chapters
- 0:00 Introduction
- 7:00 If you click bad, it asks you why…
- 10:00 The shortest paths figures out the shortest paths to each state
- 17:00 The result is that you have better…

## Transcript
[0:00] Yeah, we all write tests, or at least hopefully we all write tests, but we know that writing tests is a bit of a pain, right? So what I want to talk to you today about is something called model-based testing, and it's a way to write fewer tests. And I know that sounds counterintuitive, like, why would writing fewer tests actually be a good thing? But I'm gonna show you how it's a good thing.

[0:00] First of all, consider that however many lines of code you have, you have to write tests for that code. And the number of lines of code to write tests, you know, for that code is always going to be greater than the code itself. Over time. Because as the requirements increase, the code required to implement, those requirements increase. And then for all the use cases, if you want good coverage, the lines of code for the tests need to increase.

[0:00] And so you could see how this becomes untenable, right where? The more code that you write, the more tests you have to write in order to fully test the code. And that just, you know, just blows your budget for development. And so the sad part about this is that this leads to not being able to implement features or implement proper business requirements because you're too busy, number one, coding and number two, writing all of your tests.

[1:00] And so the problem is that, first of all, we spend too much time on tests that are the easiest to write, but not the most effective. So unit tests are the tests that, you know, we would, that's the first test that we think to write. And the problem with that is that unit tests don't necessarily let us. Like, let us really dive into how the user is actually experiencing the app. And so end-to-end and integration tests, those are going to be closer to really determining problems with, you know, with how the users use the app.

[1:00] But those tests are harder to write because they're long and they, you know, you're. You're not testing implementation details, you have to actually test a real system under test. So we just end up writing lots of units and the problem with us at unit tests don't really, you know, they don't really help, they just show that you're. A developer than those how to code. So what I mean by writing fewer tests is, I'm just posing the question, what if these tests can be generated, and what if they can be generated without being written?

[2:00] And so you have all these test cases, and let's say requirements change, so what if those same tests can be regenerated without actually being rewritten? And so this is the idea behind model-based testing. Maldis testing is not a new idea. It's like dates back to, I think the earliest mention was 2002 2005, so fairly recent. In the grand scheme of things. Things in software, but still it's been around for a while, like over 15 years.

[3:00] And so what model-based testing is? It's it's about taking a model and generating tests from it, instead of writing every single test out. So let's analyze what a test is first, or at least a good test. You know you're, you might recognize cucumber tests the given one, then style. So essentially, you have a precondition where you know you're in some sort of state of things. For example, you're in a login page and then you have a Whene, which is an action.

[3:00] So let's say it's when the user logs in, and then you have a post condition, which is the user should see a logging in page. So you have those three things and that constitutes an entire test. So what we could do is we could turn this into an abstract model, basically a boxes and arrows diagram of how the user sees the world. Where you're in this precondition, if I do this action, I will be in this post condition state.

[4:00] And then what we do is we turn those into executable tests. Because the abstract model alone is not going to be able to do anything. It just shows how things should be. And so if we map the precondition in the post condition to tests that say, all right. Actually check that this is in the DOM, or check that this is visible, or check that this is happening. And then we map the actions to actually click this button, or fill out this field, or do this.

[4:00] Then we have a real functioning test, and typically, when we're writing in ten tests, this is something that we manually write. So the. Idea is to create an abstract model, map them to real events and real assertions in the real system under test and generate them automatically. So we could represent our applications using finite state machines, which is essentially that so if you think about your application as a finite state machine.

[5:00] It's basically a linking together of all these given one, then, given when, then, so all of these chains. And so a full end-to-end test just becomes a chain of given when, then, given when, then, given when, then, all the way through. So hopefully you could sort of see how this idea of other generation comes into play. So let me explain it a different way. First, you create the model, and this model is a finite state machine that describes your.

[5:00] In this day of an app, the user could do this, or the user could do something else. And you know, you just assert that. Or you just outline all the different states that the user could be in, and how the user could get to all of those different states. And then you generate abstract tests from them. And we're going to see that. We're going to be using algorithms like death for a search, shortest path algorithms in order to do this.

[5:00] Then we make those tests real by actually mapping, remember mapping the precondition, post condition in the actions to actual things that happen in the app. And so once we do that, and once we generate all these tests that go from one state to the final state and go through all of these different paths. Then we could see exactly where tests might fail, and we could see what was the state before, what was the action that caused the test to fail, and we could identify the root cause of where tests fail.

[6:00] And then, once we actually like, go and fix the tests. Here is the kicker, because we've we've generated an abstract model. If requirements change in the future. Like, let's say, your boss comes in and says, alright. We need to add this new feature, we need to change this, we need to delete this page. Then all you have to do is update the model. You're not writing tests, you're not rewriting any tests, you're just updating the model and the tests will regenerate, and so.

[6:00] As long as you provide assertions for each of those things, your tests are going to stay up to date with your application all of the time.

[7:00] So let's take a really simple example, let's say that we have the speed back form, it says. How was your experience, good or bad? If you click bad, it asks you why, gives you a form, and if you could, if you click submit or you click good, it's. It takes you the thanks for your feedback, and you could close the form at any time. So here is an example at the Wi-Fi loads, if I click good, it says thanks for your feedback, right?

[7:00] If I say bad, it takes me to the forum. I could fill it out and then I could click submit. Thanks for your feedback. I could press the X at any time. So, you might think that this is simple, but there's actually many different ways that a user could interact with something like this. So here's what we're gonna do, I'm gonna show you how we could do this. And make model based testing using three libraries React testing, library, just and XState.

[8:00] And of course, it doesn't matter which framework you use, as I'll show you in a minute. So when making a normal test, you know, we would have our precondition. Like, let's assert that we're in the question screen, our action, which is click the good button, and an assertion that we're in the post condition. Like, we're in the thank you screen. This is what constitutes a test. This is great now you have to write like a million of these in order to fully cover your app.

[8:00] So instead, let's take a look at this as a finite state machine. When we're in the question state, we could do a number of things. We could click good, which takes us to the think screen, we could click that, which takes us to the form, or we could click close, which takes us to the closed screen. And once, once we're on form, you know, we could close from any screen. But when we submit, that also takes us to thanks.

[8:00] So we could represent this as a finite state machine over here, and this is using XState. If you're not familiar with XState, I highly recommend. You see my talk tomorrow, so if you want to get tickets, yeah, it's sort of a. But basically, I'm representing all these states over here. In this object where I have question form, thanks and closed, and I just map these events to what the next state is.

[9:00] So, question on click good, so if I click good, it goes to thanks and we map it out in this object. Now. The cool part about this object is that I could go ahead and copy this and let's see if this works. If I go here. I could just paste that exact code into here and be able to visualize what that state machine is directly from the code. So I could click good and I could see that it goes to thanks, like, okay, bad, see that it goes to form.

[9:00] If I click close, that goes up close, if I click things, it goes there.

[10:00] And so this. This creates this finite see machine that represents our app. Okay, so we're going to be using algorithms that you don't have to implement, and I'll show you why in a minute, these algorithms, since we're describing our app with this abstract model, which is a directed graph. We could basically determine how to get from one state, the initial states to every single other states, which is exactly what we want to test.

[10:00] For example, we could find the shortest path from A to E, which would be this going from A to C, and then C to E. And so this, you know, uses Dijkstra's algorithm, for instance, which means that we could also use weighted paths for this as well. Now there's a simple path, which is a death first search, which simple paths basically mean. We are trying to figure out every single path that goes from one state to another state without screeding cycles.

[10:00] Because we're going to assume that our user isn't silly enough to go forward, back, forward, back, forward, back, like a lot of times, and create a cycle. So here's one we go AC E, or a BD E, or a BCE, or even a longer one, AC PDE. Now this is going to give you full coverage. Because we're trying to figure out every single possible way that a user can use the application.

[11:00] All the possible paths a. User can take both the happy paths and the edge cases as well. So if we look at this, I have a library called XState graph, and there's a get simple paths function in there. And or get shortest paths on, or get simple path. But the shortest paths figures out the shortest paths to each state. And actually, this should be get simple past, because we're figuring out all of the different paths to get from each state.

[11:00] For example, thanks, we could do two things to get two thanks. We could. On the question screen, we could click good, which takes us to the thanks screen, or we could click bad, which takes us to the forum. And then we could click submit, which takes us to thanks. And you could see, there's many possible paths that could be generated. Do you trust yourself or your QA person to figure out all these paths?

[12:00] I wouldn't put my money on it because, you know, it's gonna be a lot of paths. So using. There's a library that I created on top of XState, called XState test, which allows you to use a state machine. The exact same state machine that we created. To represent the abstract model, we could add tests, assertions for all of the states, so, for example, if I'm in the question states, I could assert that.

[12:00] By. This is using React testing library, there's also View testing library. It's going to be the same API that, you know, we pass and get by. Test ID, question screen, and so I have, like, a data, test ID equals question screen. And so I'm saying, as long as that's there, I know I'm in the question screen and so you can see, it's the same thing for all of the other ones. Like, I am asserting that I'm in the form screen over here, and I'm asserting that I'm in of the things screen over here.

[13:00] So these are not full tests, they're just assertions for every state. Now, once I have my model, what I could do is I could map events to it as well. So my click good event right now, it's an abstract events. It's not doing anything. But I could tell my testing model. The click good events means that I'm clicking a button that has the text. Good, and you know, same thing with pressing escape, that means I'm pressing the Escape key, doing close means I'm pressing the close button, all of these sorts of things.

[13:00] And also there's a way to represents many possible ways that you could do those events, like if I'm typing different values in, etc. All right, so once I do this, I. I basically do a one-liner. To generate all these paths, so I don't have to implement the algorithms myself. I just say, give me all the shortest path plans or all of the simple path plans, and I rate through them. And so I know this is a brief description, so I don't expect you to fully read the code, but instead I'll give you links to the readme later.

[14:00] So we test each path that we generated and based on the rules that we gave it inside of our model. It's going to go through each one of those paths and make sure that each of those passes traversable. And so the results of that is that these tests are going to generate all of these different paths. And we're going to make sure that every single way a user could get, from the first dates to every single other state is fully tested, and so in this simple app, we have 27 tests generated.

[15:00] And so it's cool. Because if I, if I forgot to disable a or two, if I accidentally disabled the button, or I forgot to have a close button handler. Then these some of these paths are going to fail, and you're going to know exactly where they fail. So we could take this further and go not just React. Testing library, but we could use Puppeteer as well as just an XState to actually automate these tests.

[15:00] In the real run browser and Microsoft recently released Playwright's, which allows you to use this in WebKit browsers and edge as well. So with puppeteer, instead of those normal assertions using testing library, now we're just mapping them to actual puppeteer actions. And these tests can be async so they work well with libraries such as Puppeteer. And so in this, in this recording, you'll see that there is an actual puppeteer instance. That's running through all the tests, doing every single possible thing it can - this feedback.

[16:00] Form and asserting that it could go through every single one of those paths. And so this isn't limited to any framework, you could use this, and you React Angular, even normal frameworks as well. And and so I really encourage you to check out XState tests as well as XState. I also have an article out that shows you exactly how to use XState test with a React application. But it applies to any other application.

[16:00] And there's similar tools out there as well, such as simulate O, which was created a few years ago. But it's the same idea of model-based testing and graph Walker as well. In case you want to get inspired by other libraries doing sort of a similar thing. So, I'll be honest, like, doing this is harder because there is a learning curve. You have to learn state machines and statecharts if you wants to, and you have to.

[17:00] You have to learn how to model your application so that you could fully represent it in that state machine.

[17:00] But the result is that you have better, faster and stronger tests because you have a lot of resiliency to requirements changes, and that's the most important thing. I mean, you know how painful it is to write just one end-to-end test, now writes 20 end-to-end tests manually? And then have requirements change on you, and then you have to change all of these tests. Like, that's, you know, it's gonna be really painful. So that's why generating these tests is really a better way to go.

[17:00] So I, yeah, that's XA test as part of the X8 family, which I have. Doc's over here for you, XTS, other works, stocks. And we also have a community at spectrum about chats, last day charts. So basically, this is a way for you to make your code model-driven so that you could generate tests and even go further, generate docs and prototypes and basically make your code do more. So. Thank you [Applause].