I recently gave a talk at the St. Louis Angular Lunch on GulpJS. While not directly related to Angular, GulpJS is a great utility to have while working on frontend development, and it very popular among Angular developers. This talk explains the basic usage of Gulp and provides some minor comparisons to its largest competitor, Grunt.
Here is a video of the talk, from our YouTube channel.
Transcript
Here is the talk, transcribed to text. This is a lightly edited, draft transcription, so any errors are probably from that process.
We're going to talk a little bit about GulpJS. So, try to avoid going the route that almost everyone else takes, when talking about GulpJS, of doing the Gulp vs Grunt debate. So I'm just going to try and take the approach of talking about GulpJS as if we'd never gone down the route of talking about Grunt before.
Unfortunately, Grunt has existed for a while before Gulp showed up. So, when talking about Gulp it'll be almost inevitable to make a few of those comparisons. So kinda bear with me. As we go through it I'll try and not pick on Grunt. When learning Grunt, I didn't enjoy it. While using Grunt, I don't enjoy it. Learning Gulp and using Gulp I liked it, so I'm a little bit biased there.
This is me, I'm Paul.
What is Gulp? Gulp is a streaming build system or task runner. You use it for all kinds of things. You can compile down your Jade and your Sass. You can compress your images. You can minify your code, concatenate your code, and you can run all kinds of random tests. You can run your automated tests. You can run end tests.
The reason why we're talking about it is because you can run pre-minification steps for Angular apps. You can run all kinds of other build tasks. You can basically run any kind of code that you could run in any kind of node application, basically.
So, let's take a step back here and actually go visit the Gulp website and see what they have to say about it. So they make a sales pitch here about why they think you should use it. They like to talk about it as being code over configuration. And this one of those points that, when you're talking about Gulp, you almost can't help but compare it to Grunt.
One of their big sales pitches is that, when you're writing Gulp files, you're actually writing code. You're not trying to type up a configuration file that's loaded up and then…. when Grunt executes, reading your configuration file, it's doing what it's supposed to based upon that. With Gulp, you're actually executing a series of commands. So, that's one thing that Gulp does a little bit differently than Grunt.
Another thing that is different about Gulp is that it actually uses streams to process a batch of files. It also has a very small API. We're actually going to cover the entire API in this talk. It's got four functions in the entire API. And another thing is the plugin infrastructure. It actually has monitoring of plugins that go into it, so it's actually, to get published in their plugins library, you’re going to have to get a little bit of testing, a little bit of review in there, so they don't just accept whatever is submitted. A little bit better library for that.
So, I mentioned that Gulp uses streams. Why is this a big deal? Why does it matter? So, let's consider this example. This example, though not directly relevant to Gulp, is just a reason why streams are useful in general. This is a sample of a very, very simple http server, where they are loading up a data.text file on request, and serving it. Real simple. Now, imagine data.text was a 5 megabyte file, and you have a couple thousand current users requesting this file at once.
What's going to happen is, this 5 megabyte file gets loaded entire into memory, before the request even begins to be served back to the users. So, the user is waiting round for the 5 megabyte file to get loaded into memory, before they even begin to receive notification that the request is being handled. And then you've got this entire 5 megabyte file, sitting around in memory, for all several thousand users, for the entire duration of the time that the users, sitting on their slow dialup connection, waiting for it to be downloaded. So the entire service is going to get bogged down with all these memory requests.
Right here, same amount of code. We've just replaced reading the file in the memory, with reading it into a stream and serving it. So now, as the file is read from disk in chunks, it’s being sent straight to the user. And now, instead of loading up the entire file, they just send a little bit at a time. The memory is being freed up as it's being sent to them. You're only holding as much resources as you need at a time. The user's seeing the response as it's getting read. So you get immediate response, less resources being held on to, and you have a more responsive server. So, in general, streams are looking pretty good.
More relevant to Gulp, you've got the fact that you're batch-processing a bunch of files. Let's say you've got a build process that requires half a dozen steps, and you've got 150 source files. That's over 700 different files that you're going to end up writing every time you change a single file there in your development cycle. So, it's going to be much quicker, using these streams.
Let's talk about the Gulp API. Four calls. Source, dest, task, and watch.
Source. gulp.src, supplied above, matches the file pattern. From there, start piping on commands to various plugins. So, the return of source is a set of files. And the return of a pipe as a result of that operation, is again a set of files that continue to pipe on. Pretty simple.
Participant: Is it a set of streams?
Paul: Yes, sorry. It is a stream of the content of those files. So, yes. It is the opposite of source. It is where pipe will put those files. So here, with source, we are obtaining files from here. In this example we're piping them to a plugin called Jade, we're then piping those into this destination. Dest also happens to return the stream after it finishes placing them in that destination. So we can, in addition to placing them in .build/templates, we can then continue to pipe onto them as well. So after we've placed them here, we can continue to pipe minified and then continue on.
Task. Task is where you actually do stuff. This is how you actually kick off the process of calling these minified tasks and these Jade tasks. So, in this example, gulp.task, you provide a name, and in this case they've provided a callback function. And down here we have our default task, and they provide a list of dependencies that are other tasks within your Gulp file. We'll talk a little more about those in just a moment.
So, what can you do in task? Basically anything that you can do in any node application. In this example here they're actually using defer to write up some promises even. For this build, they’re actually trying to create a dependency that is synchronous, be you could use it for any purpose that you want.
So, in Gulp, they actually execute all the dependencies of a task asynchronously. So we go back and look at this list here. If they were to add three, four, and five onto this list, and have those listed here as well, they would all be executed asynchronously, provided that they didn't have any dependencies of their own. So, if you have a list of tasks, and you have a task that depends upon them, all of those tasks are executed at the same time.
Now, why is this both a strength and a weakness? Well, it's a strength in that you end up with really fast, concurrent processing of a lot of files. A weakness is that it actually kinda makes for a little bit of overhead when writing up your Gulp files. So let's take a look at what that entails for us.
Well, that means that we have to actually provide annoying hints to Gulp to kinda develop this synchronous behaviour in our task list. To do this, we can provide hints to our tasks in one of three ways. We can either tell our task to execute a callback, return a promise, or return a string.
Take a peek at the documentation for this. So this is straight from the Gulp documentation. Here we are providing a hint of it calling a callback. Normally, you would just end up having your task form some word, and then piping the dest, and then being done. Here they're actually returning the string that they use. And then we actually saw this example before, where they create the defer object, and promise from it, and they return that promise.
And now, down here, we have our default task, which depends upon test 1 and 2. Now, like I said before, normally 1 and 2 would be kicked off at the same time. But, because 2 depends on 1, 1 will be forced to execute first, followed by 2. Again, the only reason this works is because we provided a hint here in 1, in the form of calling a callback. So it's a real annoying (in my opinion annoying) nuance of Gulp, that you have to plan these little hints that allow synchronous execution of these tasks. It is what it is. It's a gotcha that a lot of people run into when trying to execute these tasks synchronously.
A reason why you actually don't want these things executing asynchronously, for example when you want to process your Jade files, and then turn around and start to do your pre-minification for Angular, would be an example of that.
So that leaves us with the last one, watch. Watch will look at a file pattern, and if any files in that change, it will set off this list of tasks. And again it will run those asynchronously. It will also run this particular function right here for you.
Let's talk about plugins.
Participant: Back on the watch, I think in Grunt there's no easy way to tell it that when one of those files changes, only run the task on that file. JShint, for example. Change any JavaScript file, run JShint on every JavaScript file. Is this similar to that, or does it only run on the files that have been modified?
Paul: So, this is it, right here.
Participant: Right, but is the uglify task smart enough to only run on files that have been modified since the last time that…
Participant: I think that's going to depend entirely upon the [utility group?]
Paul: This gulp.watch unfortunately doesn't. There is, however, a plugin called gulp-watch, that is smart enough to… and I will actually show an example at the end that uses that plugin. That leaves a pipe open in the middle of a task that will watch all the files var, and if it detects any changes of any of those files, it will run that individual file back through the pipeline.
So I can actually show you an example of that.
This is some of the plugins that we actually use. And you can probably guess right away from the names what they do, but I'm just going to quickly run through them. Jade for compiling Jade to HTML, minified CSS, minified HTML.
Jade Sass, for Sass compilation. ng-annotate. We just recently switched from ng-min to ng-annotate for pre-minification of Angular HTML. And then, uglify and jshint … do exactly that.
I checked last night, there are 605 plugins for Gulp as compared to the 3,000 something for Grunt. That's a bit of a gap there, but, again, Gulp has a bit of a monitoring system for the plugins libraries. Do you guys know if Grunt has any sort of monitoring or can anyone submit to it?
Participant: The ones that are blessed have the word “contrib” in their name, and there's a star next to them, and bla-bla-bla.
Paul: Okay. And I don't actually know the bar of acceptance for Gulp, I just know that that's something that they tout highly as being something that… That there's supposed to be some level of acceptance there that goes on. So I just know that that's something that they pride themselves on.
Participant: At this point I've only ever used those Less plugins. Is there — like we were talking about with Grunt, you've got the contrib, which is the… And then you've got everything else — is there an "Everything else" anywhere for Gulp other than the things you write yourself?
Paul: It can be anything that you can find on npm.
Participant: So there's nothing stopping me from putting a plugin out there if I wanted to, but it not going through their approval process?
Paul: Right. You can find their blessed ones on their website, but you go about getting them the same way you get any other node package, just using npm-install. So yeah, you could npm-install and you just use require, and you'll see that here in a moment. You can just use require to get it into your Gulp file, so yeah, you could mistakenly require something that's not even a node package in an attempt to use it.
So you could require something that’s not even a Gulp plugin and try and use it, and who knows the result for that. Same thing with trying to go out and get something that they put up, and not even attempted to submit it to the Gulp plugin maintainers. Yeah, that's definitely possible.
Alright, so let's take a look at the file here that I… This is a really, really simple one. Doesn't use half of the examples I just gave, but I kinda want to give a brief runthrough, show everyone what it looks like.
So to start off with, you actually list all of your plugins, give them a simple name. We've already talked about Jade, Sass. Watch is the plugin I mentioned earlier that with allow you to watch whatever comes through the pipe. If any of those changes, we'll rerun it through the pipe. Any of those files … we'll rerun that file that changed, though the pipe.
Plumber is used in conjunction with watch. If something were to happen further down the pipe, and an error occurs, it will still allow you to capture that error while keeping the pipe open. So it will print out the error and it'll keep the pipe open for watch to send any future changes back though.
So, actually, I'll show you an example of that in a moment. So, right here, we've got our gulp.task that will look though my Jade folder for any Jade files, send them through the plumber, through watch, finally into my Jade plugin, and stick the results into my www folder. This will simply send the same thing, send them through a watch for my CSS files, into the Sass plugin, put them in my www/css folder, likewise, JavaScript's files copy straight though. The reason why this is so simple is because I'm actually… This is the Gulp file for project that was my demonstration, so there's not much to do here.
And then my images I just pass through as well. Finally down at the bottom, we have the Gulp test that actually puts all of these together, and this is the one that we'll actually call from the command line. So, execute this and just say Gulp dev.
It picks up the Gulp file that is in here, which you name gulpfile.Js, and it automatically knows to look for that file. It will look here for this name, based on the task that you supply. So I supplied dev. Looked at my dependency list. It said, “Okay, so I need to kick off HTML, CSS, JS, and images, which it did. It's running these asynchronously, because I don't have any dependencies listed for any of them, so all these are running at the same time. It found one file, one file, three files and eight files.
So to answer your question, you were look for a solution where, if I change just one file… Let's see here…
Participant: Did it find eight JavaScript files?
Paul: … change in JavaScript files.
Let's change this one. That will break something, but I'm done with my presentation, so… Here, let's change this. So now if I save this, we should see that this will pick up the change. Sass complained, but says just that one file was changed, Sass complained about my one file that changed. And it left the pipe open for future changes, so now I can…. Actually no it didn't, because I didn't have plumber in that particular task.
So if I go here and add plumber, save that. Now it complained about it, didn't care, kept running the pipe. I’ve got my pipe open. Now if I come back here, fix my broken Sass file, save it, saw that it was changed, kept my pipe open.
Participant: So in your Grunt file at the bottom, I'm assuming if you would've named that last task default instead of dev, then you would just run Gulp …
Paul: So, I'll just name this default, save it. Here's Gulp… default task.
Participant: Was there any reason for the… callback there at the end of the…
Paul: No, there's no reason for that at all.
Participant: Not required? Maybe you have to pass it.
Participant: Is there any good reason to run watch without plumber?
Paul: Run watch without plumber… Yes. If you're doing something that you know is going to break. So right here, where I'm moving a file from A to B, if there are files. There's no process in the middle that's going to emit an error. So if I add a file or remove a file, then this particular version, gulp watch, can also detect when files are added or removed, which the built-in watch cannot. So, this one will then pick it up and chuck it over into my www/js folder.
Participant: I guess I was wondering, would it make sense to just create your own watch that calls plumber and watch, and then you don't have to call plumber. You don't have to remember to type pipe-plumber pipe-watch in each one of your… Everything you want to watch. I was wondering if there is any downside to putting plumber in something? Is that going to hurt anything?
Paul: Is there a downside to putting plumber in something? Not really, although the overhead of typing it.
Participant: For that matter, it might as well be a different version of source, that always pipes in plumber and watch.
Paul: Yeah, and I'm sure that's a matter of maturation. That's the biggest argument against… That's really the only argument I've heard against the usage of Gulp as of today. A lot of people say that it just still need to brew a little bit longer before it beats out Grunt.
Participant: That was the thing that was keeping me from using it for a while, but it's so much nicer to deal with than Grunt that I've given up that fight.
Participant: So one of the main features I like about Grunt is the live reload feature of the watch. Do you know if Gulp has an equivalent?
Paul: It has it, and unfortunately I didn't have time to get a good demo of that in. It does have that.
Do you think you could do that on the spot?
Participant: Just telling me it works is enough.
[Crosstalk]
Participant: Yesterday in the class I'm like "Hey, you guys have moved from Grunt to Gulp. Can you just throw up on the board everything you're working on? Tell us all about it." And he's like "Okay."
Paul: I actually stripped those lines of code out of this file…
Participant: So you can change any CSS, HTML, JavaScript file and it will do whatever it needs to, and then it will refresh the browser.
Participant: Yeah. There's actually a plugin, like, gulp-connect, I think it's called. It runs the server for the livereload and handles livereload. So all you have to do is run gulp-connect on whatever port you want.
Paul: Alright guys, that is it for me. Any questions? Any other questions? No. Alright. Thank you.