The following describes the asset pipeline for preparing images, JavaScript and SCSS/CSS for production. This pipeline tooling is platform independent, meaning that there shouldn't be any issues setting this up for your Mac/*nix/Windows environment (though I've been able to test this on windows yet).^[1]

There are a few key technologies that allow us to automate the process of compiling assets, primary among them is the task runner Gulp.js, which we can script to process our project's assets. Gulp is built on the server-side JavaScript runtime node.js, which is tightly integrated with the package manager NPM. NPM is where we obtain the various modules required to process our assets. For our pipeline, these dependancies (other then Node and NPM) are installed locally to the project, so they shouldn't conflict with any globally installed versions you may have.

Low level requirements

As per the convention of Node and NPM, the required dependencies [including Gulp], are maintained and organised in a package.jason file, which should be kept under the project's version control. As npm installs and updates dependancies based on package.jason there is no need to install all the parts independently –– npm makes getting up and running as painless as possible.

Quick start

Overview of Project commands

NPM scrip(s)

Gulp custom commands

Key files & directories

Once the project dependancies have been installed, the following is a breakdown of the key files you will find in the resources directory. Some of these will have been created during the npm install process.

.editorconfig plugins can be found for all major editors including jEdit and Atom.

Adding resources

This section will focus on the adding of JavaScript resources, as we use special tooling to make it easier. This said, the process pipelines listed in the following sections help illustrate the process by which assets are consumed. The concept of 'entry points' is key to understanding what should go where. Simply put, an entry point is a file which imports or otherwise links to additional resources. A .scss or .js processing pipeline doesn't have to render every file in a file tree but instead the processor looks for entry point files in a specified directory, and then follows any chain of import('bar.scss') directives to combine all linked files into a bar.css file.

/src/styles/*.{css,scss}

The watch directory for our s/css pipeline is: /resources/src/styles/*.{css,scss}

Any css or scss file found in the /resources/src/styles/ directory will be processed. The processed css file will be found in /resources/dist/styles/bar.css.

/src/js/*.{js,jsx,es,es6}:

The watch directory for our JavaScript pipeline is:/resources/src/js/*.{js,jsx}

Any *.js, *.jsx, *.es, *.es6 files found in the /resources/src/js/ directory will be processed. The processed JavaScript files will be found in /resources/dist/js/*.js.

/src/js/overleap/*.{js,map}

We've discovered that some dependancies within the project use conventions that throw errors with Browserify. For these instances, we've created the src/js/overleap/ directory. Any *.js files placed here will be copied without alteration to /dist/js/*.js.

This in this way we can begin to take the distribution directory /dis/**/ out of version control.

Note: It is highly recommended that you use pre-minified versions of these files, and where available, associated *.js.map for better browser debugging.

But wait!

JavaScript ES5 doesn't natively have import('baz.js') functionality! While the concept of programmatic loading of modules was brought ES6+, much of our codebase isn't there yet. In fact, to take advantage of ES6 modules would require("intentional-pun.js") a lot of refactoring. In our JS pipeline we use a tool called Browserify which implements for regular JS, a familiar import api as you would use with CSS. Unfortunately (at least for migration ease) the authors of ES6 took a different, though more powerful approach, so the Browserify approach is not the equivalent to ES6. Until the projects js codebase is fully brought up to modern standards, Browserify will be a core deployment dependency.

Browsers Targeted

Currently the selected browser market profile is defined using the browserlist tool, which creates a shared target list for tools like Autoprefixer, StyleLint, and Babel.

This targets browsers that command greater then .2% market share, the last two versions of those browsers, Firefox ESR (Extended Service Release, used by a lot of governments, schools and other orgs), and IE 10. Lastly, the ruleset also omits any browsers that are marked as 'dead'.

SCSS pipeline

Scripts runs gulp cleanCSS prior to processing files, which deletes the contents of the dist/styles directory.

Note: Style CSS/SCSS entry points are only gathered from the top level of /dist/styles/.

JS pipeline

Scripts runs gulp cleanJS prior to processing files, which deletes the contents of the dist/js directory.

Note: JS entry points are only gathered from the top level of /dist/js/*.js.

CSS:

Images:

Other & Linting:

package.json component overview

Known issues:

As the developer is not responding, I have created a patch and pull request, which we are using in the meantime.

This manual assumes a basic familiarity with use of the command line and common operations such as cd, ls and mkdir. ↩︎
If you need to get npm and Node installed on your system, npmjs.org has some good tutorials: setting up your local environment. ↩︎
As of this writing, node-sass is not yet compatible with the latest version of node.js v 12.0. So we recommend installing the latest LTS (long term support) version of node.js which is 10.15.3. This Stackoverflow thread covers a number of ways you can roll back to a previous version of node.js. Of special interest is the node version manager. ↩︎
We've chosen to do linting via npm rather then the gulp pipeline, for a variety of reasons, but due to these issues and general maintainability concerns, we will just use the well supported parent projects via npm rather then gulp wrappers, because it-just-works™. ↩︎ ↩︎
--prod flag can be applied to any build command ie gulp css --prod and gulp js --prod. Doing so will cause the output to be minified etc. In the case of styles, they are additionally passed through Bless to circumvent selector number limit imposed by IE > 9. (Pub trivia: IE9 has an upper limit of 4095 selectors, after which it silently fails). ↩︎

Project Resources - Build Pipeline

Overview