At our September 9, 2015 Angular Lunch, Mark Katerberg compared the different testing frameworks with AngularJS 1.x. He shows and explains the differences between Jasmine, Mocha and associated libraries, and how they compare for Angular application unit testing.
If you don’t have time to watch it, here is my super-high-level summary:
- Karma+Jasmine is a solid, very usable default. It is also what we use here at Oasis Digital frequently.
- Mocha and its associated libraries are more popular for server-side testing, and have certain advantages if you are willing to learn and adopt them.
- QUnit is most appealing to those already familiar with an older generation of testing tools, but also has completely acceptable features for modern unit testing.
But don’t take my word for it. Watch the talk video, Mark has compared all of these and more.
We have transcribed the talk about to text, provided below. This is a rough, first-draft transcription, so any errors are probably from that process.
Thanks to Oasis for hosting this. It’s always awesome. It’s good to have these things. Also, neteffects, thanks for the pizza.
So, we’re going to be talking about Angular JS testing frameworks today. There’s a lot of them. It’s a lot of complication and a lot of choices. They all kind of do the same thing, so that’s kind of going to be the end result. No matter what you pick, you’re probably going to not make a terrible choice. So, that’s the good news. The bad news is that there are a lot of choices, so you’re going to feel like no matter what you picked, it’s going to feel like the wrong choice.
Who am I? I’m Mark Katerberg. I started developing in 2010. I moved down here in 2011, and I’ve been doing Angular JS for the last couple of years. My background’s in TDD, and I’m really a big fan of test-driven development. I feel like it’s really hard for me to write code if I’m not testing it, just because then I’m not really sure it works.
I don’t mind Angular JS. I’m not as fully behind it as everyone else seems to be. I’m not sure it’s the perfect framework yet, but it seems like a pretty good one. And I’ve been doing it for a while, and it seems pretty good.
There’s all my contact information. Feel free to hit me up in any of that stuff. Twitter’s probably the easiest one, but anything’s fine.
To start off… What we’re not going to cover. This isn’t going to be an intro to Angular. There’s going to be some Angular knowledge assumed. You don’t have to be a rock star or anything, but there’s going to be assumptions of you know what a controller is, you know what a service is, you know what scope is, things like that.
We’re not going to be talking about testing directives. That’s a whole other mess, and this isn’t really an intro to testing… period. It’s a comparison between testing frameworks, not teaching you how to write tests. There will be some of that, but as far as the complex scenarios and directives, that’s not going to be part of this. Hit me up afterwards, though, because any of these things I care a lot about and love talking about with anybody.
We’re not going to talk about $httpBackend because that’s a whole other can of worms about whether you should use that or not.
There’s a million testing frameworks. We’re only going to be talking about the three big ones today: Integration Testing, API testing, using the Shockroom, and ways to interact with the browser. That’s not what we’re going to talk about.
There’s a lot of other good testing units, like JUnit, that aren’t part of this. There’s all sorts of things we’re not talking about today.
What we are going to talk about is unit tests. These are specifically automated unit tests. I think what defines unit test — there’s the things on the left.
- They’re automated.
- They’re compartmentalized, so they don’t try to do too much. They just test the unit of functionality.
- They’re fast, which means you can run it, and every time you modify a line of code, it just fires them off and makes sure everything is still working.
- They’re mobile. That means that you can move them around. If you’re moving functionality of code around pretty easily, you’re not like, “I want to move this function to another service, but that’s going to slow me down and take half a day.” No, that’s not what a unit test should be. A unit test should be easily able to move with the code and not make your day worse.
- They should also be assertive. It’s easy to write a unit test to get your code coverage up that just runs over code and doesn’t actually verify anything at the end of it. That’s not really what you want to be doing because unit tests are there for you to make sure that your code is doing what you think it does.
- Lastly, they’re something to drive your design. You really want to have your unit tests there to make your code more readable, make your code easier to use and for other people to use as well. It’s not just something to write after the fact, after you have a 20-line long function, you’re trying to unit test that, because you’re going to hate yourself for trying to do that.
What isn’t it.
- It’s not clicking on a website. That’s what we have QI people for and why we have automated browser tests.
- It’s not verified entries in the database. Once again, these are integration tests.
- They’re not instructions for humans to follow, although lots of people will tell you that’s a unit test, these are not automated unit tests in the sense that we’re talking about.
- They’re also not a Code Coverage report or 200 lines long. That might be something your boss wants, and Code Coverage is great if your boss never sees it, but as soon as it becomes something that you’re trying to achieve, it often becomes, “I’m going to get this coverage up for the sake of having coverage,” and you’re not actually getting the benefits out of the coverage. It should be there to show you what things you missed, not there to brag about and say, “I have 100% coverage,” because that doesn’t really mean anything. I can show you afterwards how you can get 100% coverage without making any assertions, and it will never fail.
History lesson about the different frameworks. The three big ones we’re going to be talking about today are QUnit, Jasmine, and Mocha. As you can see, QUnit came out a long time before this. That’s actually it’s official release date, but QUnit was part of the JQuery library long before it was ever its own separate product. It’s the way they handled making sure they didn’t have to rush it into their code.
They literally came out about a year apart from one another. Jasmine came out and then Mocha came out right afterwards. Their popularity: Jasmine is the pretty clear winner as far as gitHub stars, and generally just the activity on it seems to be pretty far ahead of Mocha’s as far as testing frameworks go. They both far outstrip QUnit as far as who actually wants to use these things. That’s what you see in the industry too. It’s like everyone uses Jasmine unless they have a strong reason not to, or unless they’re very strongly opinionated and like to argue about things.
Jasmine is the default that people use unless they have a good reason not to. I think if you asked me one year into doing Angular, “Why do you use Jasmine?” I wouldn’t have an answer for you.
The functionality they all provide.
They’re all a framework. I know that can mean a million things. That’s just a buzzword. But fundamentally they all are ways to structure tests. You write a test and they know how to find the series of tests and interact with them and run your tests.
That brings up, what is the test runner? The framework is how you write your test, the syntax for how the tests actually look in relation to one another. The runner is the thing that says you have 25 files across your file system, and when you change a section of code, I am watching all of those and guarding your file system, and I will fire them off and actually run all of your tests to make sure they’re all working.
So the framework is actually the tests themselves. The runner is the thing that makes those tests go. All of them share Karma, so you can use Karma across any of them, which is what we’re going to be doing in the example today.
Mocha has it’s own built in one. I haven’t played with it personally at all, but it seems to do what it says on the tin. It runs your tests. It does just fine.
The next layer we’re going to talk about is assertions. These are the things that actually say one should equal one. Like when I ask this controller to do something, it should have done that thing. What it gives me back is what I expect to have back.
The assertion libraries for QUnit and Jasmine are baked right into the framework. Mocha, however, has a modular way of doing assertions. So, there’s all sorts of different libraries. The big three seem to be Chai, inspect.js, and should.js. These are really the big choices once you choose Mocha. These are the things that actually influence how your syntax looks. Whether you say I expect this to equal this, and here’s the message you should display otherwise, or whether you say scope.foo.should.equals scope.bar. It’s the syntax for how you make your assertions. We’ll see this in examples later on. So, don’t worry. This is all pretty technical and not necessarily important to understand. There are choices that Mocha gives you because it’s more of a compartmentalized framework.
You can see that in your mocking as well. The fundamental level of mocking is that, when I have a controller that talks to a service and gets functionality back from that service, I don’t want to have to test that in the controller. Because if this service uses another service uses another service, and that one is shared across about 18 different controllers, if I change that behavior, it will break all 18 tests you have across your controllers.
The way you prevent that is by mocking out the service calls and saying I want you to display what this service tells you; I don’t care what this service is actually doing. If you know what I’m talking about, it makes a lot of sense. If you don’t know what I’m talking about, don’t worry about it, we’ll see that in the examples as well.
The important thing is that Jasmine, as you can see from this, is pretty monolithic. It has everything baked right into it other than the runner. The other ones have a bit more modularity with Mocha being by far the most modular. Mocha’s very much a plug and play framework as far as testing.
That has benefits and downsides. The benefits are I can use exactly what I want to use for my team. My team can argue about something and decide on what they want to use, and that’s great. But it also means that if somebody starts writing something using should.js syntax, and somebody else is writing it in Chai, your code suddenly doesn’t look the same, and there might be some confusion about why is this working here, why isn’t it working here? Why are we using different things?
Jasmine is very prescriptive. The upside of that is that now everyone can use the same code, and everyone can read everyone else’s code.
Now we’re going to jump right into examples. Are there any questions on this layer here? It’s not super important if it’s mind boggling and doesn't make any sense.
Ppp: You might be getting into this later, so if you want me to wait…
Ppp: ….Do you have any sense of why this is the way it is? Why did the Jasmine people decide to make it monolithic? Is there any information out there?
Mark: As best I can see, it’s that Jasmine was created and said, “We’re going to create the best version of testing and mocking. The best everything. Here it is, presented to you on the stone tablets of the Ten Commandments.” Then Mocha saw that, and over the course of the next year said, “There are these things I don’t like about this. I’m going to change them. I’m also going to give other people the option to change them if they don’t like my choices.”
So, it’s very much the OSX versus BSD approach. Like there was example of here’s a system that works perfectly if you abide by the rules of that system versus I want to be able to customize every possible thing to this system.
And QUnit is created back when it was created. And they didn’t think about mocking. There wasn’t a need for it if you’re writing JQuery functions that interact with the browser. If you think about the logic for how QUnit was created, it was created that when I click on this button, I expect this functionality to change over here. Because that’s what you use JQuery for. There’s not really a massive MVC framework inside of JQuery. You can do it, but it wasn’t something you’d think about in 2008.
I think historically QUnit was built for a specific purpose and has been extended from that point. Jasmine saw that and said, I want to have a better way, and here it is. Then Mocha was a reaction to Jasmine. I think the historical context is that, but I could be completely wrong. That’s just my feeling about it.
Mark: The examples are all up on this repository. Feel free to check it out. There will be a link at the end as well if you want to copy it down later. But if you want to follow along, or inevitably we run out of time, we can see where to go from this point.
We’re actually going to look at some examples of code. Let’s just show you the file system here. Karma is our test runner. Karma has a million different plugins for it that do things like reporting. They can show you how your test failed or passed in different ways. They even have a Nyan Cat reporter if you want. You can also have it watch your file system for you or just run once and not run it. So these are configurations that you can use for things like production where you don't want it to sit there and watch your file system, but you do in development.
You could have it refresh your browser every time you made a change to your file. If you use Ruby, it’s very similar to Guard. It just watches your file system and does things when things change. In this example, we have these three different Karma configurations. We have our Jasmine configuration, our Mocha configuration, and our QUnit configuration.
If we look at those, they’re all pretty similar. We’re going to walk through the Jasmine one first, and then talk about the other two as well.
You define at the top, here’s my configuration for how Karma should run. Our framework we’re going to use is Jasmine in this case because we’re running our Jasmine tests.
Then the files we need. Whenever you add a new library, Karma needs to know about that as well because it stands up your project, but it also needs to have all the libraries that you need to run your project. In this case, we’re referencing Angular. We’re using angular-mocks, which is a way of mocking out the underlying Angular libraries. This is things like, when I want to create a controller, I need to use angular-mocks to create a controller for me.
Then there’s one file is this case that we’re testing, is the controller.js file. That controller is just for the sake of this example. You’re obviously going to have gulp or something building out these files for you, but in this example, we’re just talking about testing this one file.
The last item here. It’s going to look for any file that ends with JTest.js. You can call your files whatever you want. I like to have my test files sitting right next to my code files because that means when I change something it’s just right there. I don’t have to look up a giant directory and back down another one to find the same package name or whatever you do. It’s all right in the same place. That’s why we have compilers, gulp and things to move our code and only take the pieces we want.
We’re not excluding anything. The port it binds to is here. Logging and the browsers. So, you could have this run in PhantomJS which is by far the fastest, but also sometimes gets weird and buggy. I’ve not seen this happen in the last two years, but there was one project where we were doing a lot of directives and actual manipulation of the DOM that the PhantomJS just reacted differently than Chrome or Firefox did. So you could have this run across a bunch of different browsers.
This is a lot bigger deal when you start doing integration tests and things like that that aren’t unit tests.
Then Single Run true or false. Those are so you can have it watch the file system or not. In this case, we’re just going to have it only run once. So, we’re going to have to run it each time.
The other examples: Mocha is very similar. The only difference, instead of Jasmine, we’re running Mocha and we have our mocking framework of choice there. This is both our mocking and assertion framework. Remember that Chai at the top was one of the assertion libraries, and Sinon is one of the mocking libraries.
There are half a dozen different mocking libraries. For that slide, I just tried to simplify it down and show the big ones that people actually use on a day to day basis pretty regularly.
The last one is the QUnit. Here’s the one difference: We’re using QUnit and our tests start with QTest. All golden.
To actually run these tests, you do karma start and then pass the configuration file. So, if we want to run our Jasmine tests, you just do that.
So, if we want to run Jasmine tests, you just do that. Once you execute l, you get a bunch of gobbledygook and then it says, "Executed 1 of 1 – success." If there were failures they would show right there.
If we now look at our controller.Jtest, this is the test that actually ran because we told it to scan for anything that starts with .jtest. This is all the boilerplate you need to run to write a test. It's a lot. That's one of the big downsides to testing in Angular is that there are a lot of different things you have to do to set things up.
It is what it is. It's nice that you only run it once, and then it's generally 95% the same across all of them. If you want to keep things compartmentalized and not interacting with one another, this is what you've got to do.
The controller we're going to be testing is right here. This probably looks familiar to you, if you've written controllers before. I'm not using gulps. There's no ng-annotate or anything to make this look simpler.
Literally, we're standing up a module called testApp. We're binding a controller to it called TestCtrl, and we have a scope on the controller. That's all this does.
Our test for that controller — the only one we have — is called it(‘is defined’). Let's walk through what this is talking about.
Describe is like a grouping that says that everything inside of me is going to be referring to this thing. So, its TestCtrl is the thing that it's referring to. Then, before each test that runs inside of this block, we're going to stand up the module testApp.
So, that says we're going to find the module testApp and have it there. beforeEach just runs before every test that isn't there.
Then, we have another block that says ‘after start up’. This might seem a little confusing at first because it's a controller. Of course it's after startup. Sometimes if you're doing things at controller startup like when you startup, you're going to fetch a bunch of data and have it run immediately, and it's not going to inject it into you. You might need to do things before the controller stands up.
That's why we might have another block in here that's something like ‘before start up’. Then we could have other tests inside of there where we actually run this controller startup procedures inside of each test individually. For this example, we're not going to be talking about that.
But that's the reason we have an ‘after start up’ block.
Then we have another before that actually does the injection of the controller and $rootScope. This is very similar how in Angular when you inject something, you define — I'm going to have an injector on this. Here are the things that I need. It's going to pass them into you.
Angular Mock handles a lot of this stuff for you, so it's going to find the controller, pass things to you like that.
Once we have those things, we're going to create a new instance of $rootScope so we're not interacting around things. We're going to add semi-colons because style is important. And we're going to create a controller called test controller.
Instantiate this thing and use this $scope object to do so.
All crystal so far?
Ppp: One thing that I want to point out to people… I moved from QUnit to Jasmine when I first started using Angular. Therefore my first exposure to Jasmine was Angular tests with Jasmine. Therefore, I have no freaking clue what was Jasmine and what was not. The module and the inject come from angular-mocks. But beforeEach and the describe and the expect and the it, that's all Jasmine stuff. I wish I had known that sooner.
I just like to point that out to people. If you've never seen Jasmine before, module and inject are Angular things that have been added to Jasmine. All that other stuff is Jasmine.
Mark: Correct. And what he's talking about there is specifically this library right here handles a lot of those things to allow us to get access to the Angular items that we can test. This will become obvious when we look at the QUnit version of this. It will look moderately different.
Ppp: I can add a point closely related to Bill's. So, code like this looks pretty different from production code in many ways, but the one that really strikes me most is that the dependencies seem to be coming in from everywhere. We've experimentally detected that there's a difference ethos in test code than in production code in that it seems to be just part of what the community does, which is dump all the tools you might need into the global scope. Whereas that would be highly frowned on. Can you comment on that?
Mark: I don't like doing that. It is necessary is some cases like when you pull in Jasmine, it drops everything onto there.
Ppp: Right here. This code right here.
Mark: So, you're saying controller and things like that? What are you saying?
Ppp: I'm just reading down the list. So, describe, beforeEach, module, inject, and it. You just have to know they're on the global scope. They're not nameSpace. There are no requires, you don’t have to go get them from somewhere…
Mark: That's fair. I think that's just for convenience sake. This isn't code that's getting deployed at production. It's not like this is terrible. You would frown on it for production, I agree. But it's not like if you put something and make it accessible to everything below you… It is what it is. I think that's just kind of the way that people think about writing test code.
Ppp: I'm really eager to see — probably in your next slide — if that's a Jasmine thing specifically or if that same pattern follows in the other tool. Eager to see.
Mark: It's not. So, the last thing is to digest scope to make sure nothing weird is happening on scope from other tests. So this is just a catch-all that you throw on top of them. The only test we have here is that it is defined. So, that just says, scope exists essentially. You could change the syntax to be undefined, and then we run over here and tests are failing because we expected this object that had all these things on it to actually be undefined.
So, this syntax here… This is Jasmine. Everything else, the module injector is Jasmine, and the inspect is Jasmine. And the actual framework itself of the it, described, beforeEach, but the actual assertion library — that's this part right here.
So, as a way of comparison, let's look at the Mocha tests because it looks very similar. So, this is a Mocha test. You'll notice that it definitely came after Jasmine. They steal a lot of the same syntax from them. We still have a beforeEach. We still have a describe. All of the logic here is the same. You have nested blocks for describes, and you have its for individual test runs.
The only difference between those two right here is that they're using the Chai assertion library to say should.exist(scope). So, should is the item on scope. There's a million different ways that you can do this. You could also be like expect scope to equal or to not equal undefined. That would be another way you could do it. That's just a different Chai assertion.
Let's go look at the weird beast. QTests looks very different. It this case, you create a test by saying test and give it a name. Then there are lots of different things that, once again, QUnit drops onto scope. So, okay is one of them. Expect is another one. But the actual initialization for them is a little less hierarchical. Remember those describe blocks that show that these things are all being run before this? Inside of QUnit, you instead create modules and modules have initializers, which is right here. So, this initializer has a set up, and you can also have a tear down. I think the new syntax calls it beforeEach, so they're kind of falling in line with what everyone else is doing.
These are essentially the same thing as that beforeEach inside of a describe block. A module does need things to stand up, and we're using Angular.injector to receive these objects rather than using the angular-mocks to give them to us.
Then we're setting up a controller in the same way. We $scope.$digest.
Ppp: That module on line 122… Is that QUnit's module–
Mark: That's QUnit's module. It is saying… All tests that run after this one are part of this module, so they all need the have these before.Eaches and the initialization happened for them. So, if we create another test right after here that does … right? So both of these tests will run with that before.Each before them. So, it'll run the before.Each, then it will run the test. Whereas this one up top here doesn't have anything run before it because it's not inside of a module. Does that make sense?
Ppp: The module thing…now you can see why I was totally confused when you went straight from QUnit to Angular…Parts of Jasmine.
Ppp: Do you have to say Angular.Module to get Angular's module?
Mark: Exactly. Which is what you see up here on line 8. We have an Angular.injector. When you're using QUnit, you call out Angular items specifically.
Now let's check our tests. They're all running properly. I made some aliases this year just to make things a little easier. kJasmine, kMocha, and kQUnit. Everything's not passing. Because I named variables really. Cool. So everything's passing.
So, let's do a simple test to say we're going to have a controller function, and our controller function should bind whatever we pass to it onto scope. If I call scope.electItem and I pass it an input… This is going to be something, then…
Just to clarify, I'm certainly not an expert in QUnit or any of these libraries. I feel pretty good about my knowledge of Jasmine, and okay about Mocha. They're all very different, so i'm going to be consulting this thing right here. So, QUnit does a syntax like this. scope dot selected item because we're going to follow a pretty asinine example here. So, we expect scope.SelectedItem to equal our input. And if it doesn't equal the message we're going to give back is, "You didn't … item on scope."
So, if we run our test, we get a message back that says it died on test one because undefined is not a scope function when it tried to call scope.SelectItem.
If then we just go over to our controller code, and we say, I obviously forgot to create my select item function. There we go. Perfect. The function should pass, right? Oh no. You didn't put the item on scope. How strange for us.
We have one failing test. Let's try to do that same test over here. Actually, let's pass the test first. Then we can do the same test over there. For this example, scope.SelectedItem equals input. Boom. Now we have the tests passing.
We now are guaranteed the scope method is going to put the thing where it's supposed to put it. Really simple example.
Now let's comment that out and see how things fail in other versions. For the syntax here, we could just create another it and describe what it should do here. I like to wrap all of my functions calls in their own describe blocks just to say what — just to clarify the chain of when they fail. And we'll see as soon as it fails why I do this. So, scope.Select item input a thing.
Ppp: Is it legit to do that directly inside a describe?
Mark: No, it's not. I totally missed that. Thanks. It binds item to scope. These are terrible test names, so… I can be flamed by my team later. That's fine. So, expect $scope.selectedItem to be input. So, there's a lot of different things you can say here. I can say I expect it to equal input, which is a soft equals versus a hard equals, like double equals versus triple essentially. You can also have things like, to be undefined. Or to be defined. There's all sorts of different… You can check out the Mocha documents for that.
Of course, I thought I was in Jasmine test, but I’m not. $scope.selectedItem.should.equal is the syntax for this one. It decorates every item with a should that then Mocha knows how to attach to subsequent things. So you can say scope.item should equal input.
Sorry about that. I didn't realize which file I was in.
Ppp: We didn't either.
Mark. No, it's fine. So, kMocha. Now it's saying undefined is not a function because we don't have that. Let's jump over here, make it a function and see how it fails now.
The nice thing about writing out in describe blocks and being overly verbose in your test names, is that now when it fails, we see test controller after startup select item function binds item to scope failed. It reads off in an English readable way so you can figure out where your tests are failing and what they're supposed to do.
This is why test names are really important because you're able to track down from a failure what it actually should be doing, and that's… Your expects and assertions are valuable but really the test name is the most valuable thing because it tells you what the business logic should be. It tells you something like when I select an item, I expect it to query against the service and show the message. It actually tells you what it should be doing. The assertions are there as backup really.
In this case, undefined is not an object because when it's trying to evaluate ‘$scope.selectItem.should’. The reason for that is because we obviously haven't created something on scope. If we now create it, it passes. If we suddenly said typo, set it equal to input, run it, and now it says insertion error expected input to equal a thing. Then we can go into the test and figure out what a thing is and debug that way.
It's nice that it catches you in certain ways.
Now, the last one is Jasmine, which is going to be super similar to this Mocha test, so I'm actually just going to steal all this for the sake of time. Because the describe and it syntax is the same between those languages, often you'll find yourself — if you're transitioning — the transition's a lot easier that way than it is to go to QUnit where it's just a very different way of thinking about how tests are structured.
In this case, instead of saying $scope.selectItem should equal, we say we expect that's one of the things that dropped onto global scope of what expect means. We expect $scope.selectItem to equal or to be — in this case it's be because we expect it to actually bind it. Now we can do kjasmine and tests are passing.
Does that make sense? Any questions so far? Cool. So “failing over there because it expected undefined to be a thing. But the syntax for where it finds off, it used the describe and it syntax, it reads off just the same way. You see your failure messages right here in line. Cool.
Now we can talk about an example where we're using a service. We can do something more than just binding item onto scope. Let's actually try grabbing a service. We only have five minutes, so it's going to be a little quick. So here we go.
This syntax is probably terrible, so apologies, but I'm just going to grab some for the sake of time. So we're using to same module. We're not going to instantiate it for each one of them because that's bad. This one we will. I'm dropping everything into this controller file. That's also bad. I do lots of bad things.
So we create our module. Then our controller's going to bind to that module and our service is going to bind to that as well. So, I'm creating the boiler plate before I write the tests. The tests in driving your development don't necessarily always have to be written first. You just have to be thinking about how am I going to test this before you start writing your code. If you write all your code first, and then try to test it, often it becomes a nightmare, and you end up not covering things. And you get this massive spaghetti web of things, then tests that cover them in name only.
It's a factory, so we need to return a thing. Try to run it. Things are running. So, they're able to run over it. It doesn't care that there's a test service there. That's not going to matter for our controller tests. But let's get a describe block for our test file. Actually because all of this is just — we can do js/service.jtest.js. And once again copy and pasting because I'm a bad person. This describe. We keep those always the same. It doesn't really matter. This is just for readability. So we're going to make test service be the thing worth doing here.
We're also going to create an instance variable up top that we have access to. We don't need to worry about after startup because in this case, we're not doing anything with startup. We don't need to care about scope or even rescope. So, for injections, for services, they just get injected directly in. For controller we had injecting the $controller and instantiate it. For services, we can just do something like testService. The underscores around it are completely just syntactic sugar. That way you can do something like testService equals test service. That means you can expect that you expect to use elsewhere. In this case, we're just going to call it instance because that's convention I like.
Syntax is hard. So, now we have another test that tests that service is instantiated correctly. We can now write another test that — we're just going to talk through this because there's not much time. We're going that have another test that asserts that the service interacts the correct way. And if you check out the gitHub project for this, you're going to see there's a lot of different commits talking about what happens when I have a service, I test it only using controller code, and just don't bother testing the service but I test the controller's use of the service.
In that case, what would happen if the service changes, it breaks the controller test. That might be good or bad, but if it's used across five different controllers, it's going to be really bad. It's going to be really frustrating to have to track down this changed, so now you need to change five different tests. That's where something like mocking comes in. In the examples in that repo, we talked about how mocking can help alleviate that pain, and not really have to run into that problem. Also, in there if you want to see an example of how you can do injection of services and controllers for testing, there's examples across all three different languages as well.
The big thing I try to take out of this is that when you are testing services, make sure that you are testing the actual functionality of that service, and that the controller doesn't duplicate that test. Right, so the controller shouldn't care how the service works, it should just care that it calls it and uses whatever it gives back. This is kind of the basis of how mocking works in testing.
So I know we didn't really get a chance to kind of talk through all of the implications of how different they really look at that kind of level. But, I think from the high level you can kind of see how close Jasmine and Mocha are. It's essentially just assertion libraries are slightly different. And then, mocking for Mocha uses Sinon, versus the Jasmine build in mocking.
Whereas QUnit you can do all the same things, it's just a little bit, it just looks very different. So if we look at our QUnit test there, it just– Qtest, there we go. It just looks a little different, right? If you're used to Jasmine or Mocha this just looks a little strange. You can totally do all the same things, it's just there are, you can just walk into weird pitfalls where you're not used to it.
So I would say there's definitely no testing framework that's right or wrong, they just all provide the same functionality in different ways. Jasmine is the most popular, so that's honestly when I walk into a new team, the reason why I pick it. It's not that it's better than Mocha or better than QUnit, it's just that more people are familiar with it and it's nice to have a prescriptive guide of how people should all write their code the same way. It means that you don't have to worry about somebody starting to use the BDD syntax that Mocha gives you versus using the Expect JS syntax.
But, yeah, that's really it. I would encourage, if you're interested go check out the repository and see the more complex examples of how you can actually deal with interactions there. Any questions about testing in general? What's up?
Participant: So, you know, your subtitle at the top was "Which ones suck and which ones suck differently."
Participant: And from teaching people testing, I've seen this general resistance to it.
Participant: So, what's next? What are people doing to lower that bar that you've seen, of the next round.
Mark: So, there's not a lot of good solutions. I mean, the biggest thing, I think that helps people is just getting into the mindset of thinking about testing first. Because if you've– written your code then it feels like, "Oh, man i've got to write these tests, that's a huge speed bump to getting released, and there's no real point to them, they never catch anything for me." And yeah, that's totally true if you do that, because you're not writing your tests to help you, you're writing your tests for somebody else.
You shouldn't be writing your tests for your boss. If you're doing that then you're doing it wrong. Yeah, you can do it and just make some other people happy and make yourself slightly less happy, or from the beginning you can spend the time learning. Actually like, "Why are these tests valuable, what should I be testing that I think somebody else is going to screw up and I'll care that they screw up."
So, when i'm writing tests, really that's the first thing I always talk about. I want to catch the next person in the things that I think they're going to, that I care that they don't change. So, if I have some functionality that doesn't matter to me, then first of all, I probably shouldn't be writing it, but if I am, i'm not going to write a test around it, right? Because somebody else changes that, then they don't need to know why I wrote it.
Whereas, if it's something that I wrote that I think actually has a reason to be written, then I think that there needs to be a test around it, because I want them to know when they change it, "Hey, you are changing this, someone else used to care about this thing." And maybe they don't any more, that person is probably going to be me in six months and I'll have forgotten by that point.
And that's why it's nice, right, because in six months from now, when I change something, i'll be like, "Oh, that's right, I had that conversation with this person and they told me this rule is necessary. We can't just simplify this code this way."
And I think that the biggest thing is not so much we need a better tool, or less complex syntax. The biggest thing is just kind of getting into the mindset of thinking of tests as something for you, versus a test that is something for somebody else.
Because if you have a hundred-line method and try to write a test for that, you're going to hate yourself for a long time. If you start writing a test at the beginning, before you write the code, or write things in little sections, where you write a little code, write a little test, and go back and forth. You're going to find yourself being like, "I need to extract this, this is too much in this one test. There's too many cycle paths in here. I don't want to have to test these all at once. I'm going to move them out somewhere else."
And you're going to find your code getting more modular, and you're going to find more terms for reuse, and your code is just going to look better and you're going to feel better about it afterwards. And it will make you thinner and more attractive too. What's up?
Participant: I’d like to be thinner and more attractive. So, about the thing you were talking about like two minutes back. So, if I'm happy with Jasmine, because it's kind of, not literally in the box, but it's kind of all there…
Participant: There seem to be people out there who are pretty passionate about Mocha…
Participant: So, what are they excited about?
Mark: So, Mocha has a lot of really cool stuff it does. It provides, I think a better assertions– set of assertions. And by better, I mean more broad. You can write things in ways that actually make sense to a business user. Like if I talk to an analyst and say, "I expect this value to equal this value." A lot of times that doesn't really click in their head. But if I write things in ways that are readable as English, a lot of times your assertions can just become your business rules, and become like, "These are my acceptance criteria and I expect that this item that is in this array should include these other items as well." And it becomes English readable.
But the assertion libraries are only one layer to it. Where Mocha I think really shines and outstrips Jasmine is actually on the back end. It's not what we want to hear because this is Angular, but Mocha has a lot of really nice interactions that require js and as that stuff moves into the front end, which it's doing, I think that Mocha might have a better case for why it should take over.
But it does a lot of better things with being able to deal with injections and things like that, that I'm not really using on the front end yet. I know other people are, so I'm sure that's probably why they're feeling that pain. It's because using requires with Jasmine I've heard is a nightmare. Never had to deal with that pain myself, but if I did, I'd probably be pushing Mocha a lot harder.
Participant: What I keep hearing is that if you're a node shop you're already up to your eyeballs in Mocha.
Participant: There’s no use fighting that tide if you're also going to be adding Angular to the mix, because everything you can do in Jasmine, you can also do in Mocha. You might have some initial choices up front you have to make, like which file library you want to use, which mocking library you want to use, but once you've made those decisions, you're certainly at parity with Jasmine, and you're not using a different assertion library on the front end and from the back end.
Participant: But like we were just talking about with Paul, I usually find so much friction from people who are like, "Eww, testing, that's like you forcing me to eat my vegetables." I don't want to give them any additional friction about "You must choose your mocking library, you must select–" No, use Jasmine, all the decisions are made for you, and at the point where you're like, "I have exhausted the capabilities of Jasmine." Then you get to reevaluate the decision. And that will be like, 10 years from now for most people.
Mark: And the nice thing about that is because they are so similar in structure, it's often just a matter of, "I need to sub out my spies, I need to sub out my assertions." It's not that hard to migrate.
Participant: Along the lines of what Paul was asking about. All the boilerplate with Angular, that really sticks, all that friction I was just talking about, just like, "Testing is great and you really should do testing. Oh, and by the way, here's 97 lines of boilerplate that you have to do in front of every Angular test that you do."
Participant: Matt and I were talking last night about things like ng-describe, and some of the additional matcher libraries for Jasmine. Have you used any of that stuff for real editing in quantity? Because I don't really know what to recommend at this point to make that easier.
Mark: So, ng-annotate is great. Ng-describe is really useful but can catch you if you're doing really complex stuff. Ng-describe is great if you're kind of writing tests and writing code the way we were writing it right here. Where you have a controller and it’s binding directly to this, everything is kind of in one place, and if you don't have a lot of modules that interact in weird ways. Ng-describe I've seen have some trouble with that stuff and it's probably just that the configuration was set wrong.
But the trouble was, right, I didn't want to spend a week figuring out the configuration for ng-describe. I do think it's useful, it's just not something I want to become a crutch for myself, because I find the configuration step more painful then copy-and-pasting the boilerplate.
Participant: It sounds like you landed in the same place that we did.
Mark: The one thing I do think is super nice that I know is in the works right now is World Wide Tech. I used to work there for a while, we wrote a library to kind of help with some of the mocking, and when you give it an object, it gives you back everything with methods all mocked out on it. And they're working on open sourcing that, so that should probably happen in the next few weeks. It's called Fragrance. Whenever that happens, I'll send it out to the group. But, that's one that I've found very useful and I really hope I can start using it again.
Participant: What is it called again?
Mark: It's called Fragrance. It's not out yet. So, they're still going through a legal–
Participant: Can you spell that?
Mark: Fragrance. Like a smell. Like Jasmine smells like stuff. Yeah, it's a terrible pun, just awful.
Participant: We do need to wrap it up.
Mark: Yeah, the one last thing I'll say is that Jasmine's great because everything you need to know about it is in this one page. Right, it's literally a giant sheet that gives you examples of how to use every possible thing in the framework. You can start at the top, read a quarter of the way down, and start using 90% of it. It's just like, if somebody has a question about Jasmine, you point them at this one page, whereas Mocha you have to look across a bunch of things. That's the big advantage of Jasmine in my mind. Kind of echo your sentiment. I don't think that it's better, it's just that it's all in one spot. It does what it says on the tin, and it's easy. That's why I use it.