Thursday February 5, 2015 - tags:    node.js, gulp, browserify, sourcemaps

Gulp - a streaming build system

Gulp is a streaming build system built on node.js. It utilises node streams enabling; in most cases a fairly fast build process. Plugins are written using node.js, using code over configuration. Search npm for gulpplugin and gulpfriendly to see the already large selection of gulp plugins.

Gulp is a rather elegant solution to setting up a build system. Gulp tasks work with a file(s) source, the files at that source are read into memory which returns a stream, the stream is piped through a series of gulp plugins which transform that stream, the result of which is saved to a destination source.

This post will demonstrate setting up gulp tasks for a typical web application development workflow. This includes:

The source code for this post is here:

https://github.com/AndrewKeig/gulp-tasks

Gulp API

Lets look at the Gulp api; it basically uses 4 methods to constuct a build task:

gulp.task registers a task by name and allows you to sequence dependencies to be executed before the task runs, these will run in parallel.

gulp.task('default', ['dependencies'], function() {  
    ...
});

gulp.src reads file(s) from a node-glob pattern into memory and returns a readable stream.

gulp.task('default', function() {  
    return gulp.src('src/*.js');
});

gulp.dest returns a writable stream to a destination folder.

gulp.task('default', function() {  
    return gulp.src('src/*.js')
        .pipe(gulp.dest('public/js'));
});

gulp.watch watches a glob pattern and runs a function on file change.

gulp.task('watch', function() {  
    gulp.watch(['src/*.js'], ['build']);
});

Setup & Run Gulp

First install gulp globally

$ npm install --global gulp

You should also install gulp to devDependencies in your package.json file

$ npm install --save-dev gulp

You can run the default gulp task with the following command

$ gulp

You can also run a gulp task by name

$ gulp lint

Setup a default task

Gulp tasks are stored in a gulpfile.js. We can create a default task using gulp.task('default', ['dependencies']) which registers a task by name and specifies a list of dependencies to be executed before this task runs. Here he specify all of the tasks we would like to run as part of our workflow. We need to run these in sequence, i.e. not in parallel so wrap this with run-sequence, which ensures tasks do not run before the previous one completed.

Below we are using require-dir which allows us to split up gulp tasks into separate files, requireDir('./tasks') will pull them all into our main gulpfile.

'use strict';

var gulp           = require('gulp');  
var requireDir     = require('require-dir');  
var runSequence = require('run-sequence');

requireDir('./gulp/tasks', { recurse: true });

gulp.task('default', function(callback) {  
  runSequence('lint', 
    'test', 
    'browserify', 
    'html', 
    'css',
    'server', 
    'watch' , callback);
});

Lint

Lets create a lint task, that allows us to lint our entire codebase. We start by defining a gulp task called lint by calling gulp.task('lint' fn). gulp.src(['sources']) which reads file(s) from an array of sources into a readable stream of objects representing those files. The stream is piped using pipe() to jshint(), which performs linting using rules defined in .jshintrc file. Jshint has multiple reporters, we pipe the stream to jshint.reporter which reports the results.

'use strict';

var gulp   = require('gulp');  
var jshint = require('gulp-jshint');

gulp.task('lint', function() {  
    return gulp.src(['app/components/*.js', 'test/*.js', 'gulp/tasks/*.js', 'gulpfile.js'])
        .pipe(jshint('.jshintrc'))
        .pipe(jshint.reporter('default'), { verbose: true })
        .pipe(jshint.reporter('fail'));
});

Test

The following task test uses Mocha to run all tests located test/*.js. The call to gulp.src() takes an options object, {read: false}, which simply means, do not read the file content into memory.

'use strict';

var gulp  = require('gulp');  
var mocha = require('gulp-mocha');

gulp.task('test', function () {  
    return gulp.src('test/*.js', {read: false})
        .pipe(mocha({reporter: 'min'}));
});

CSS/SASS

The following task css, is fairly complex, we read all files of type sass,scss from ./app/css/*. First we rename the file, ading a min suffix. We would like to generate sourcemaps so we pipe to sourcemaps.init(), which in turn pipes to sass(), which takes an options object {outputStyle: 'compressed'} and compresses the output. autoprefixer() will add vendor prefixes to CSS rules for the last two browser versions. We then write the sourcemaps to location './maps', which is actually written to the folder './public/css/maps', finally the css files are written to './public/css`.

$ gulp

HTML

The following task html is simple, we read our html files from './app/html/*.html' and pipe to minifyHTML() which minifies the content. We also use replace90 to add some version information to the page title and write the html files to './public'.

$ gulp lint

Browserify

The following task browserify, will browserify our scripts with minification and sourcemaps. gulp-browserify is no longer required and we can simply use the various vinyl packages; browserify has its own streaming api.

Note: this is actually the preferred course of action if a module has a streaming api.

We start by creating a browserify object which takes an object with entries './app/components/app.js' a list of files to browserify and debug which enables sourcemaps. We then use source to create a text stream and rename the file app.min.js. We now pipe to buffer() which creates a stream transform, and use sourcemaps.init() to start the sourcemaps process. We then minify the scripts by calling uglify(), and write the sourcemaps to location ./maps. The browserified scripts are written to ./public/js.

gulpfile.js

Server

The following task server, simply starts an express server, and serves statics from a public folder.

default

Watch & Livereload

We would like to re-run tasks when changes are made to source. This is easily achieved with gulp using gulp.watch.

This statement will execute when we make changes to our scripts or tests:
gulp.task('default', ['dependencies']) This statement will execute when we make changes to our gulpfiles and taks:
run-sequence This statement will execute when we make changes to our html:
require-dir This statement will execute when we make changes to our css:
requireDir('./tasks')

We also setup livereload by calling livereload.listen(); and we have set livereload up to run on any code change:

gulpfile

'use strict';

var gulp           = require('gulp');  
var requireDir     = require('require-dir');  
var runSequence = require('run-sequence');

requireDir('./gulp/tasks', { recurse: true });

gulp.task('default', function(callback) {  
  runSequence('lint', 
    'test', 
    'browserify', 
    'html', 
    'css',
    'server', 
    'watch' , callback);
});

Code coverage

The following task coverage, is not included in the main default workflow, it will run code coverage with Istanbul. We start by specifying the location of our code to test, app/components/*.js. We pipe to istanbul() and add a handler for on finish. When Istanbul has finished analysing our source we run the tests at location test/*.js using Mocha; Istanbul writes the coverage reports to ./coverage, in html format.

lint

Bump

The following task bump is not included in the main default workflow, it simply reads our package.json file and will bump() the version number with minor numbers, writing the file back to root.

'use strict';

var gulp = require('gulp');  
var bump = require('gulp-bump');

gulp.task('bump', function(){  
  gulp.src('./package.json')
  .pipe(bump({type:'minor'}))
  .pipe(gulp.dest('./'));
});