ServerSide JavaScript with MooTools and Node.js

This blog post is intended to provide a starting point for ServerSide JavaScript (SSJS) development with MooTools. It is focused on Node.js (http://nodejs.org) and tries to explain the main concepts and differences from client side development. It is solely based on my current experience, my current work, and the best practices I have defined for myself so far - though most of this has been heavily influenced by people from other people of the MooTools team.

How to Setup Node.js

• Download from http://nodejs.org/#download
• Build via ./configure && make && sudo make install

It can't get any easier.

Current State of MooTools and SSJS

Our current releases, MooTools Core 1.2 and 1.3beta2, do not work out of the box with Node.js. Node.js, as well as other serverside JavaScript implementations have adopted the CommonJS standard which includes a module system. Every module you create can export objects via the "exports" object. You can include a file by using "require('path/to/module')" which gives you access to the module's exported variables:

math.js

exports.add = function(a, b){
	return a + b;
};

myApplication.js

var math = require('./math');

var sys = require('sys'); // System module

sys.puts(math.add(13, 37)); // Outputs "50"

You can execute this script via "node myApplication.js" on the command line.

You can find more information about this on the CommonJS Wiki: http://wiki.commonjs.org/wiki/Modules/1.1.

The key difference between the module system as specified in CommonJS and normal client side script-tags is that they do not share the same (global) scope. This means that creating a variable via "var foo" in a module does not automatically make it available on the global object. The global object on the client side is the "window"-object which is usually not available on the serverside. In Node.js the global object is called GLOBAL, whereas some other implementations simply use the name "global" or just reference it to "this" inside a module. This is not defined by CommonJS so every environment solves it in a different way.

While it is relatively easy to add support for modules in a JavaScript library that revolves around only one object, MooTools provides several global variables such as Class, Events, Type (Native) etc. In addition to that this new standard is very young and if we ever implement support for CommonJS directly into MooTools we want to define the best practices that we can recommend to our whole community.

Note: CommonJS is not actually a standard but more of a set of specifications that a serverside implementation of JavaScript can (or should) follow to unify the various environments and to make it possible to create modules that work on all platforms without any modifications.

Get MooTools Running on Node.js

Over the past couple of months some members of the MooTools team came up with various ways to make MooTools CommonJS compatible. I have now created a repository on GitHub that helps create a build version of MooTools. This is mostly based on work by @keeto and me. We are going to use the work-in-progress version of MooTools Core, which is a pre 1.3 version. If you don't have git installed or don't feel like entering some commands you can skip ahead to the next section and just download a pre-build version of MooTools: MooTools.js (based on this commit).

Get MooTools 1.3wip (command line)

git clone git://github.com/cpojer/mootools-core.git

Get Packager (requires php-cli to be installed)

git clone http://github.com/kamicane/packager.git

Get the MooTools CommonJS Loader

git clone git://github.com/cpojer/mootools-loader.git

Build a Custom MooTools Version

cd packager # Switch into the Packager folder

./packager register /path/to/mootools-core
./packager register /path/to/mootools-loader

./packager build Loader/Prefix Core/Class Core/Class.Extras Loader/Loader -blocks 1.2compat > MooTools.js

You should see some output like this:

Build using: Core, Loader
Included Files/Components:
- Loader/Prefix: [Prefix]
- Core/Core: [Core, MooTools, Type, typeOf, instanceOf]
- Core/Array: [Array]
- Core/String: [String]
- Core/Function: [Function]
- Core/Number: [Number]
- Core/Class: [Class]
- Core/Class.Extras: [Class.Extras, Chain, Events, Options]
- Core/Object: [Object, Hash]
- Loader/Loader: [Loader]

and a file "MooTools.js" should have been created that is ready to be used.

Use MooTools in Node.js

require.paths.push('path/to/mootoolsjs/');

require('MooTools').apply(GLOBAL);

var sys = require('sys');

var MyClass = new Class({

	initialize: function(){
		sys.puts('It works!');
	}

});

new MyClass;

Run this, again with the "node" command, on the command line and you should see "It works!" as output.

Please note that this is quite experimental, the functionality you will find is subject to change and might contain bugs. The above solution has so far only been tested on Node.js but it should work on other SSJS implementations too

Difference Between Applications and Modules

One thing that I want to highlight is that I'm convinced that modules should not create global variables. However, the above mentioned solution puts everything MooTools provides on the global scope. While it is reasonable to do this when you are developing an application and you have control over every aspect of it, I do not believe it is a good idea to do this when you create a module (plugin) that uses MooTools. This is why the solution I came up with has another way to work with it, consider the following example.

MyPlugin.js

var Moo = require('MooTools'),
	Class = Moo.Class,
	Options = Moo.Options,
	typeOf = Moo.typeOf;

exports.MyPlugin = new Class({

	Implements: [Options],
	
	options: {
		name: ''
	},
	
	initialize: function(options){
		if (!options) options = {};

		if (typeOf(options.name) != 'string')
			throw new Error("Ohmy!");
		
		this.setOptions(options);
	}

});

MyApplication.js

// Add path to MooTools module so every module can just require "MooTools" without specifying the exact path
require.paths.push('path/to/mootoolsjs/');

var MyPlugin = require('path/to/MyPlugin').MyPlugin;

new MyPlugin({name: 'Kid Rock'});

// We can still add all the MooTools objects to the global scope without breaking anything
require('MooTools').apply(GLOBAL);

new Class(..);

You can now share the MyPlugin-Class with other people and it will work for them even if they do not put the MooTools objects on the global scope.

Note: MooTools still adds extensions to the native types, such as String, Array and Function even if you do not put it on the global scope. Executing "require('MooTools')" once makes all the extensions available in any scope. Note that, at least at the moment, all modules share the exact same datatypes; there are no sandboxed datatypes. If extending the native types in JavaScript does not align with your style of coding you should probably not use MooTools (or JavaScript, as it is a core feature of the language). One of the goals of the MooTools project is to provide a framework that feels natural and does not make a distinction between the core language and functionality of the library.

Why "evented"? Async, huh?

JavaScript, as a language mostly used on the clientside, has strong asynchronous capabilities. This means that most of the time you define certain functionality in your application that gets executed when a user - a real person - interacts with the content of a website. You usually do this by adding listeners to certain events on DOM elements:

myElement.addEvent('click', function(){
	// This function gets executed upon interaction
});

You call the addEvent method and add a function to it, the program flow continues normally and the function is called asynchronously whenever a user clicks on it. In any case, you do not want to wait with the execution of any other code until this event gets executed. You do not want the click event listener to block. This is the main design goal of Node.js: to be non-blocking for any I/O operations such as reading or writing a file, storing or retrieving data from a database etc. This means that most code you will write on the serverside passes around functions (callbacks, event listeners) that get executed at a later time (e.g. when results of an operation are ready). You can find a simple example right on the Node.js website: http://nodejs.org/

To give you a better understanding, this is some sample code of what user authentication could look like with MooTools:

var Moo = require('MooTools'),
	Class = Moo.Class,
	Db = require('Database').getDatabase(),
	sha1 = require('Sha1');

exports.User = new Class({

	initialize: function(name){
  		this.name = name;
	},

	authenticate: function(password, callback){
		var user = this;
		Db.open(function(db){
			db.collection('users', function(err, collection){
				if (err) return callback(err);

				collection.findOne({name: user.name, password: sha1.hex(password)}, function(err, data){
					if (err) return callback(err);

					callback(null, data != null);
				});
			});
		});
	}

});

In your application you would now create a new instance of User and call the authenticate method with a callback like this

	var User = require('User');

	var sys = require('sys');

	var instance = new User('david');
	instance.authenticate('kidrock', function(err, result){
		if (err) return; // handle database error

		if (result) sys.puts('User authenticated');
		else sys.puts('User does not exist');
	});
	sys.puts('Authenticating user');

This example will print out two messages "Authenticating user" and the result/success of the authentication. The order relies on the speed of the database and likely "Authenticating user" will be printed out first. You can compare this to a setTimout example

setTimeout(function(){
	log('Bye');
}, 1);
log('Hello');

Note: The callback style with the error as first argument is aligned to the way Node.js currently works with asynchronous operations. Before that a "Promises" system was used but it has been removed. A high level implementation can abstract away from the current solution. Feel free to implement your own callback/event system with MooTools and share it with the community :)

Note: CommonJS actually specifies module identifiers to be lowercase. However, I like my filenames to start uppercased. You'll always see me doing "require('User')" instead of 'user' in my applications.

Why ServerSide MooTools?

One reason for the existence of JavaScript libraries is the lack of certain functionality especially on the DOM level and the enormous amount of issues between different rendering engines. You do not have any of these problems on the serverside. MooTools, as a library, works on a much lower level than some other libraries and it therefore provides useful utility functionality that we think is missing in JavaScript. In addition to that, even if there are CommonJS specifications, some implementations differ from the others. MooTools Core and an abstraction layer on top of that (like Deck ( http://github.com/keeto/deck )) can greatly benefit you and help you eliminate or reduce low level problems you may encounter at some point during development.

In addition to that the sophisticated and clean class system provided by MooTools makes it possible to write abstract plugins that will work on both the server- and the clientside without any further modifications (ie. a Language/Internalization class, a schema validator, etc.). For more information on this feel free to watch my presentation at FOSDEM 2010 (at 25:00 + Q&A).

Other (non-MooTools Related) Stuff

• Driver for MongoDB which I used to contribute to: http://github.com/christkv/node-mongodb-native
• WebSocket Server for NodeJS by MooTools Developer Guillermo Rauch: http://github.com/LearnBoost/Socket.IO-node
• ORM on top of the node-mongodb-driver by Nathan White and Guillermo Rauch: http://github.com/nw/mongoose
• Other Node Modules: http://wiki.github.com/ry/node/modules

Bonus: Execute a Script in Node.js from TextMate by Using Node.js

Add this command as a script:

#!/usr/bin/env node

var sys = require('sys'),
    spawn = require('child_process').spawn;

spawn('node', [process.ENV.TM_FILEPATH]).stdout.addListener('data', function(data) {
  sys.puts(data);
});

Like this:

Click the image above for a larger view.

About Christoph Pojer

Christoph is a student of Software Engineering and Business Administration at the Graz University of Technology in Austria. He is an experienced web developer and a Core Developer of the MooTools JavaScript Framework.

Christoph's Website • GitHub • Twitter • Forge