Table of Contents for The Sample

• • Foreword
  • Acknowledgment
  • Introduction
    • Why
    • What This Book is
    • What This Book is Not
    • Who This Book is For
    • Notation
    • Navigation
    • How to Use the Book
    • Examples
    • Contact Us
• I Quick Start
• II The Interface
• III Tips and Tricks
• IV Tutorials and Examples
  • 1 Instagram Gallery
    • 1.1 A File Structure
    • 1.2 Dependencies
    • 1.3 Node.js Server
    • 1.4 Handlebars Template
    • 1.5 Conclusion
  • 2 Todo App
    • 2.1 Scaffolding
    • 2.2 MongoDB
    • 2.3 Structure
    • 2.4 app.js
    • 2.5 Routes
    • 2.6 Jades
    • 2.7 LESS
    • 2.8 Conclusion
  • 3 REST API
    • 3.1 Test Coverage
    • 3.2 Dependencies
    • 3.3 Implementation
    • 3.4 Conclusion
  • 4 HackHall
    • 4.1 What is HackHall
    • 4.2 Running HackHall
    • 4.3 Structure
    • 4.4 Express.js App
    • 4.5 Routes
      • 4.5.1 index.js
      • 4.5.2 auth.js
      • 4.5.3 main.js
      • 4.5.4 users.js
      • 4.5.5 applications.js
      • 4.5.6 posts.js
    • 4.6 Mogoose Models
    • 4.7 Mocha Tests
    • 4.8 Conclusion
  • ExpressWorks
    • What is ExpressWorks?
    • Installation (recommended)
    • Local Installation (advanced)
    • Usage
    • Reset
    • Steps
      • Hello World
      • Jade
      • Good Old Form
      • Static
      • Stylish CSS
      • Param Pam Pam
      • What’s in Query
      • JSON Me
  • Related Reading and Resources
    • Other Node.js Frameworks
    • Node.js Books
    • JavaScript Classics

Foreword

Dear reader, you are holding a book which will open you to understanding and fluent usage of the Express.js framework — de facto standard in web application programming on Node.js. I would especially recommend this book because it was written by a practicing engineer, one who has a comprehensive knowledge about the full stack of web application development and Express.js in particular.

Azat and I worked on the same Node.js/Express.js code base at Storify — the social media curation tool that Washington Post, CNN, BBC, The White House and other news corps use — which was recently acquired by LiveFyre. Right before the Express.js Guide release, he asked me to write the foreword, because it’ll sound objective, sincere and unbiased coming from the creator of another Node.js framework: CompoundJS.

However, nobody is reading forewords. So, instead of a foreword, I’ll share my story. Actually, I never thought it was worth sharing and there’s definitely nothing exciting about it. But from the other point of view — thousands of young programmers living similar ordinary lives - it could be inspiring: it’s a common story, but a kind of successful one.

My way to web development started when I was a student and joined a team as junior PHP programmer. I was working here for about five years and the main lesson I have learned was: education is nothing compared to real work experience. The next page of my professional life was work in outsourcing (PHP and Ruby on Rails). And then I found Node.js.

It was something that I always wanted: processes that do not have to wait for DB/IO operations which are keeping all the resources, but doing something useful instead. This is the simple reason why I started using it; it’s more efficient compared to synchronous programming environments. By “efficient” I do not mean not speed of processing, but more flexibility in programming style.

As a good example of this flexibility, I can share some solutions I recently programmed for a Redis adapter for the Jugglingdb ORM. Problem: during peaks of website usage, we are running a lot of db queries to serve pages, and most of the queries are the same. The obvious solution is to cache results of the queries, but this solution requires additional coding and some logic for cache invalidation. We’ve come with a better solution: cache queries and not results. When a query comes, we don’t execute it immediately; instead, we wait for some time, collect identical queries, then execute query once and run multiple callbacks to serve all clients. This solution is simple and requires no additional logic. As a result, we have flat db usage even during peaks. This solution is natural in Node.js, and that’s why Node.js rocks!

Life after discovering Node.js was great, full of interesting challenges and work, but one thing was annoying: each time I start a new project, I have to do almost the same work to organize code. For me as a Rails developer, it was really great to be able to create well-structured MVC applications fast, generate scaffolding controllers / views and other stuff. But this kind of tool was missing in Node.js and that’s why I spent my Christmas holidays writing it; the project was called express-on-railway at first, then RailwayJS, then CompoundJS.

The main goal of this project was to bring structure to an Express.js application, add the ability to extend applications in a standard way, and generate application code. So, it was not a new framework, but just Express.js with decent MVC structure, which is good for developers who don’t need to learn anything but Express.js to be able to understand what’s going on in CompoundJS application. And it was a kind of piggybacking on Express.js and Rails experience: the idea was to get best ideas from rails and bring to node platform, and Express.js was selected as the base because it is the most popular framework for Node.js and has a relatively big community, so I won’t be alone with my “new framework”. It was the start of my open-source years, which completely changed my attitude to programming and all matters, but that is another story.

And what can I say to conclude: web development in Node.js started with Express.js. It is a minimalistic and a robust framework which gives you all you need to build decent web applications. Even if you decide to move to some more advanced frameworks at some point, Express.js knowledge is a basic skill you have to learn. In addition, this book contains everything you need to know to start using Express.js and clearly explains all concepts and answers to most of the questions that newcomers usually ask. For these reasons, this book is a must-read!

–

Anatoliy Chakkaev,

Creator of CompoundJS and JugglingDB

Acknowledgment

This book would not be possible without the existence of my parents, the Internet and JavaScript. Also, words cannot express my gratitude to Tony Phillips, Shawn Drost, Douglas Calhoun and Marcus Phillips for founding HackReactor and helping many people venture into new careers.

In addition to that special thanks to General Assembly, pariSOMA and Marakana for giving me opportunities to test my instructions; to Peter Armstrong for LeanPub; to Sahil Lavingia for Gumroad; to Daring Fireball for Markdown; to Metaclassy for Byword; to Fred Zirdung for advices; to Janet Williams for editing; and to Rachmad Adv for the splendid cover!

Introduction

Why

Express.js is the most popular Node.js web framework yet. As of this writing (September of 2013), there are no books that are solely dedicated to it. Its official website has bits of insights for advanced Node.js programmers. However, I found that many people — including those who go through HackReactor program and come to my Node.js classes at General Assembly and pariSOMA — are interested in a comprehensive resource. The one that would cover all the different components of Express.js work together in a real production-like application. The goal of Express.js Guide is to become such resource.

What This Book is

Express.js Guide is a concise book on one particular library. This book contains Express.js API 3.3.5 description, the best practices on code organization and patterns, real-world examples of web apps. The topics include but not limited to middleware, command-line interface and scaffolding, rendering templates, extracting params from dynamic URLs, parsing payloads and cookies, managing authentication with sessions, error handling and prepping apps for production.

For more details and for what exactly the book covers, please refer to the Table of Contents.

What This Book is Not

This book is not an introduction to Node.js, nor is it a book that covers all aspects of building a modern day web application, e.g., websockets, databases and (of course) front-end development. Keep in mind that readers also won’t find in Express.js Guide a resource for learning programming and/or JavaScript fundamentals.

You might want to take a look at Rapid Prototyping with JS for the introduction to Node.js, MongoDB and front-end development with Backbone.js.

In the real-world and especially in Node.js development, due to its modularized philosophy, we seldom use just a single framework. In the book, we have tried to stick only to Express.js and leave everything else out as much as possible, without compromising the usefulness of examples. Therefore, we intentionally left out some important chunks of web developments, for example databases, authentication and testing. Although these elements are present in tutorials and examples, they’re not explained in detail. For those materials, you could check books in the Related Reading and Resources section at the end of the book.

Who This Book is For

This book is for people fluent in programming and front-end JavaScript. In addition, to get the most benefits, readers must be familiar with basic Node.js concepts like process and global, and know core modules, including streams, clusters and buffer type.

If you’re thinking of starting a Node.js app, or of rewriting an existing one, and your weapon of choice is Express.js — this guide is for you! It will answer most of your “how” and “why” questions.

Notation

The code blocks will look like this: console.log('Hello reader!');. The terminal and console commands will have prefixes $ and > correspondingly.

If we mention and use a specific file or a folder, those names will be monospaced as well: index.js.

If there is a class or a module name in the text, it’ll be emboldened, e.g., superagent.

The new resources will be hot-linked only the first time we mention them.

Navigation

In PDF, EPUB/iPad and Mobi/Kindle versions, you could use the Table of Contents, which is in the beginning of the book and has internal links, to jump to the most interesting parts or chapters.

For faster navigation between parts, chapters and sections of the book, please use the book’s navigation pane which is based on the Table of Contents (the screenshot is below).

The Table of Contents pane in the Mac OS X Preview app.

PDF version of the book is suitable for printing on US Letter paper because all links are in the footnotes.

How to Use the Book

If you’re contemplating using Express.js or writing an app with it, read the whole book from top to bottom. However, if you’re somewhat familiar with the framework, jump straight to the question at hand. You can also skim through them later and more advanced chapters that deal with code organization and getting apps production.

Examples

The book contains code snippets and run-ready examples. The latter is available in GitHub repository at github.com/azat-co/expressjsguide. Additional examples are in http://github.com/azat-co/todo-express, github.com/azat-co/sfy-gallery and github.com/azat-co/hackhall.

The provided examples were written and tested with the specific versions of dependencies only. Because Node.js and its ecosystem of modules are being developed rapidly, please pay attention whether the new versions have breaking changes or not. Here is the list of versions that we used:

• Express.js v3.3.5
• Google Chrome Version 28.0.1500.95
• Node.js v0.10.12
• MongoDB v2.2.2
• Redis v2.6.7
• NPM v1.2.32

Contact Us

Let’s be friends on the Internet!

• Tweet Node.js question on Twitter: @azat_co
• Follow on Facebook: facebook.com/azat.mardanov
• Website: expressjsguide.com
• GitHub: github.com/azat-co/rpjs

Other Ways to Reach Us

• Email Azat directly: hi@azat.co
• Google Group: rpjs@googlegroups.com and https://groups.google.com/forum/#!forum/rpjs
• Blog: webapplog.com
• HackHall: community for hackers, hipsters and pirates

Share on Twitter with ClickToTweet link: http://clicktotweet.com/HDUx0, or just click:

“I’m reading Express.js Guide — the most popular Node.js framework’s manual by @azat_co #RPJS”

I. Quick Start

This is a sample. Read the full version of Express.js Guide at gum.co/express or jump to Express.js Guide packages and pricing.

In this part, we’ll briefly go over what Express.js is, install it, build a few simple applications and begin to touch on configurations.

II. The Interface

This part can be used as a stand alone reference for the Express.js API, because it contains description of all the main methods and properties, as well as short examples and explanations.

This is a sample. Read the full version of Express.js Guide at gum.co/express or jump to Express.js Guide packages and pricing.

III. Tips and Tricks

This is a sample. Read the full version of Express.js Guide at gum.co/express or jump to Express.js Guide packages and pricing.

In this part we’ll cover some common patterns for Node.js/Express.js development such as code organization, streams, Redis, clusters, authentication, Stylus, LESS and SASS, domains, security and Socket.IO.

IV. Tutorials and Examples

This is a sample. Read the full version of Express.js Guide at gum.co/express or jump to Express.js Guide packages and pricing.

1 Instagram Gallery

Example The full source code of this example is available at https://github.com/azat-co/sfy-gallery.

Storify runs on Node.js and Express.js; therefore, why not use these technologies to write an application that demonstrates how to build apps that rely on third-party APIs and HTTP requests to them?

The Instagram Gallery app will fetch a story object and display its title, description, and a gallery of the elements/images like this:

Instagram Gallery: Storify API + Node.js = <3

1.1 A File Structure

1 - index.js
2 - package.json
3 - views/index.html
4 - css/bootstrap-responsive.min.css
5 - css/flatly-bootstrap.min.css

Where index.js is our main Node.js file and index.html the Handlebars template.

1.2 Dependencies

Our dependencies include:

• express: 3.2.5 for Express.js framework
• superagent: 0.14.6 for making HTTP requests
• consolidate: 0.9.1 for using Handlebars with Express.js
• handlebars: 1.0.12 for using Handlebars template engine

The package.json file:

 1 {
 2   "name": "sfy-demo",
 3   "version": "0.0.0",
 4   "description": "Simple Node.js demo app on top of Storify API",
 5   "main": "index.js",
 6   "scripts": {
 7     "test": "echo \"Error: no test specified\" && exit 1"
 8   },
 9   "dependencies": {
10     "express": "3.2.5",
11     "superagent": "0.14.6",
12     "consolidate": "0.9.1",
13     "handlebars": "1.0.12"
14   },
15   "repository": "",
16   "author": "Azat Mardanov",
17   "license": "BSD"
18 }

1.3 Node.js Server

At the beginning of the file, we require dependencies:

1 var express = require('express');
2 var superagent = require('superagent');
3 var consolidate = require('consolidate');
4
5 var app = express();

Then, configure template engine:

1 app.engine('html', consolidate.handlebars);
2 app.set('view engine', 'html');
3 app.set('views', __dirname + '/views');

Set up a static folder middleware:

1 app.use(express.static(__dirname + '/public'));

If you want to serve any other story, feel free to do so. All you need is the username of the author and the story slug, for my gallery of the capitol of Tatarstan, Kazan, leave the following:

1 var user = 'azat_co';
2 var story_slug = 'kazan';

Paste your values, i.e., Storify API key, username and _token if you have one. As of this writing, Storify API is public, meaning there is no need for the authentication. In case this changes in the future, please obtain your sign up for an API key at http://dev.storify.com/request or follow the official documentation at dev.storify.com:

1 var api_key = "";
2 var username = "";
3 var _token = "";

Let’s define the home route:

1 app.get('/',function(req, res){

Now we’ll fetch elements from Storify API the route’s callback:

1   superagent.get("http://api.storify.com/v1/stories/"
2     + user + "/" + story_slug)
3     .query({api_key: api_key,
4       username: username,
5       _token: _token})
6     .set({  Accept: 'application/json' })
7     .end(function(e, storifyResponse){
8       if (e) next(e);

To render the template with the story object which is in the HTTP response body’s content property:

1       return res.render('index', storifyResponse.body.content);
2     })
3
4 })
5
6 app.listen(3001);

1.4 Handlebars Template

The views/index.html file:

 1 <!DOCTYPE html lang="en">
 2 <html>
 3   <head>
 4     <link type="text/css"
 5       href="css/flatly-bootstrap.min.css"
 6       rel="stylesheet" />
 7     <link type="text/css"
 8       href="css/bootstrap-responsive.min.css"
 9       rel="stylesheet"/>
10   </head>
11
12   <body class="container">
13     <div class="row">
14       <h1>{{title}}<small> by {{author.name}}</small></h1>
15       <p>{{description}}</p>
16     </div>
17     <div class="row">
18       <ul class="thumbnails">
19       {{#each elements}}
20         <li class="span3">
21           <a class="thumbnail" href="{{permalink}}"
22           target="_blank">
23             <img src="{{data.image.src}}"
24               title="{{data.image.caption}}" />
25           </a>
26         </li>
27       {{/each}}
28       </ul>
29     </div>
30   </body>
31
32 </html>

1.5 Conclusion

Express.js and superagent allow developers to retrieve and manipulate data provided by third-party services such as Storify, Twitter and Facebook in just a few lines of code.

Note In most cases, service providers (such as Google, Facebook and Twitter) require authentication (which is not the case with Storify API as of this writing). To make OAuth 1.0, OAuth 2.0 and OAuth Echo requests, consider oauth (GitHub), everyauth (GitHub) and/or Passport(website and GitHub).

2 Todo App

In our Todo app, we’ll intentionally not use Backbone.js or Angular to demonstrate how to build traditional websites with the use of forms and redirects. In addition to that, we’ll explain how to plug-in CSRF and LESS.

Example All the source code is in the github.com/azat-co/todo-express for your convenience.

Here are some screenshots of Todo app in which we start from a home page:

The Todo app home page.

There’s an empty list (unless you played with this app before):

The empty Todo List page.

Now we can add four items to the Todo List:

The Todo List page with added items.

Mark one of the tasks as “done”, e.g.. “Buy milk”:

The Todo List page with one item marked done.

Going to the Complete page reveals this done item:

The Todo app Completed page.

Deletion of an item from the Todo list is the only action performed via AJAX/XHR request. The rest of the logic is done via GETs and POSTs (by forms).

The Todo List page with a removed tasks.

2.1 Scaffolding

As usual, we start by running

1 $ express todo-express
2 $ cd todo-express
3 $ npm install

This will give us the basic Express.js application.

We’ll need to add two extra dependencies to package.json, the less-middleware and Mongoskin libraries:

1 $ npm install less-middleware --save
2 $ npm install mongoskin --save

Changing the name to todo-express is optional:

 1 {
 2   "name": "todo-express",
 3   "version": "0.0.1",
 4   "private": true,
 5   "scripts": {
 6     "start": "node app.js"
 7   },
 8   "dependencies": {
 9     "express": "3.3.5",
10     "jade": "*",
11     "mongoskin": "~0.6.0",
12     "less-middleware": "~0.1.12"
13   }
14 }

2.2 MongoDB

Install MongoDB if you don’t have it already.

1 $ brew update
2 $ brew install mongodb
3 $ mongo --version

For more flavors of MongoDB installation, check out the official docs.

2.3 Structure

The final version of the app has the following folder/file structure (GitHub):

 1 /todo-express
 2   /public
 3     /bootstrap
 4       *.less
 5     /images
 6     /javascripts
 7       main.js
 8       jquery.js
 9     /stylesheets
10       style.css
11       main.less
12   /routes
13     tasks.js
14     index.js
15   /views
16     tasks_completed.jade
17     layout.jade
18     index.jade
19     tasks.jade
20   app.js
21   readme.md
22   package.json

The *.less in bootstrap folder means there are bunch of Twitter Bootstrap (the CSS framework) source files. They’re available at GitHub.

2.4 app.js

This is a break down of the Express.js generated app.js file with addition of routes, database, session, LESS and param middlewares.

Firstly, we import dependencies with Node.js global require() function:

1 var express = require('express');

Similarly, we get access to our own modules which are app’s routes:

1 var routes = require('./routes');
2 var tasks = require('./routes/tasks');

The core http and path modules will be needed as well:

1 var http = require('http');
2 var path = require('path');

Mongoskin is a better alternative to the native MongoDB driver:

1 var mongoskin = require('mongoskin');

One line is all we need to get the database connection object. The first param follows standard URI convention of protocol://username:password@host:port/database:

1 var db = mongoskin.db('mongodb://localhost:27017/todo?auto_reconnect', {safe:true\
2 });

The app itself:

1 var app = express();

In this middleware, we export the database object to all middlewares. By doing so, we’ll be able to perform database operations in the routes modules:

1 app.use(function(req, res, next) {
2   req.db = {};

We just store the tasks collection in every request:

1   req.db.tasks = db.collection('tasks');
2   next();
3 })

This line allows us to access appname from within every jade template:

1 app.locals.appname = 'Express.js Todo App'

We set the server port to either the environment variable or if that’s undefined to 3000:

1 app.set('port', process.env.PORT || 3000);

These statements tell Express.js where templates live and what file extension to prepend in case the extension is omitted during the render calls:

1 app.set('views', __dirname + '/views');
2 app.set('view engine', 'jade');

Display Express.js favicon (the graphic in the URL address bar of browsers):

1 app.use(express.favicon());

Out-of-the-box logger will print requests in the terminal window:

1 app.use(express.logger('dev'));

The bodyParser() middleware is needed for painlessly accessing incoming data:

1 app.use(express.bodyParser());

The methodOverride() middleware is a work around for HTTP methods that involve headers. It’s not essential for this example, but we’ll leave it here:

1 app.use(express.methodOverride());

To use CSRF, we need cookieParser() and session():

1 app.use(express.cookieParser());
2 app.use(express.session({
3   secret: '59B93087-78BC-4EB9-993A-A61FC844F6C9'
4 }));

The csrf() middleware itself. The order is important; in other words, csrf() must be preceded by cookieParser() and session():

1 app.use(express.csrf());

To process LESS stylesheets into CSS ones, we utilize less-middleware in this manner:

1 app.use(require('less-middleware')({
2   src: __dirname + '/public',
3   compress: true
4 }));

The other static files are also in the public folder:

1 app.use(express.static(path.join(__dirname, 'public')));

Remember CSRF? This is how we expose it to templates:

1 app.use(function(req, res, next) {
2   res.locals._csrf = req.session._csrf;
3   return next();
4 })

The router plug-in is enabled by this statement. It’s important to have this line after less-middleware and csrf() lines above:

1 app.use(app.router);

It’s possible to configure different behavior based on environments:

1 if ('development' == app.get('env')) {
2   app.use(express.errorHandler());
3 }

When there’s a request that matches route/RegExp with :task_id in it, this block is executed:

1 app.param('task_id', function(req, res, next, taskId) {

The value of task ID is in taskId and we query the database to find that object:

1   req.db.tasks.findById(taskId, function(error, task){

It’s very important to check for errors and empty results:

1     if (error) return next(error);
2     if (!task) return next(new Error('Task is not found.'));

If there’s data, store it in the request and proceed to next middleware:

1     req.task = task;
2     return next();
3   });
4 });

Now it’s time to define our routes. We start with home page:

1 app.get('/', routes.index);

The Todo List page:

1 app.get('/tasks', tasks.list);

This route will mark all tasks in the todo list as completed if the user presses all done button. In a RESP API, the HTTP method would be PUT but because we’re building classical web apps with forms, we have to use POST:

1 app.post('/tasks', tasks.markAllCompleted)

The same URL for adding new tasks as for marking all tasks completed, but in the previous methods itself (markAllCompleted) you’ll see how we handle flow control:

1 app.post('/tasks', tasks.add);

To mark a single task completed, we use aforementioned :task_id string in our URL pattern. In REST API, this should have been a PUT request:

1 app.post('/tasks/:task_id', tasks.markCompleted);

Unlike with the POST route above, we utilize Express.js param middleware with :task_id token:

1 app.del('/tasks/:task_id', tasks.del);

For our Completed page, we define this route:

1 app.get('/tasks/completed', tasks.completed);

In case of malicious attacks or mistyped URLs, it’s a user-friendly thing to catch all requests with *. Keep in mind that if we had a match previously, the Node.js won’t come to execute this block:

1 app.all('*', function(req, res){
2   res.send(404);
3 })

Finally, we spin up our application with good ‘ol http method:

1 http.createServer(app).listen(app.get('port'),
2   function(){
3     console.log('Express server listening on port '
4       + app.get('port'));
5   }
6 );

The full content of app.js file:

 1 /**
 2  * Module dependencies.
 3  */
 4
 5 var express = require('express');
 6 var routes = require('./routes');
 7 var tasks = require('./routes/tasks');
 8 var http = require('http');
 9 var path = require('path');
10 var mongoskin = require('mongoskin');
11 var db = mongoskin.db('mongodb://localhost:27017/todo?auto_reconnect', {safe:true\
12 });
13 var app = express();
14 app.use(function(req, res, next) {
15   req.db = {};
16   req.db.tasks = db.collection('tasks');
17   next();
18 })
19 app.locals.appname = 'Express.js Todo App'
20 // all environments
21
22
23 app.set('port', process.env.PORT || 3000);
24 app.set('views', __dirname + '/views');
25 app.set('view engine', 'jade');
26 app.use(express.favicon());
27 app.use(express.logger('dev'));
28 app.use(express.bodyParser());
29 app.use(express.methodOverride());
30 app.use(express.cookieParser());
31 app.use(express.session({secret: '59B93087-78BC-4EB9-993A-A61FC844F6C9'}));
32 app.use(express.csrf());
33
34 app.use(require('less-middleware')({ src: __dirname + '/public', compress: true }\
35 ));
36 app.use(express.static(path.join(__dirname, 'public')));
37 app.use(function(req, res, next) {
38   res.locals._csrf = req.session._csrf;
39   return next();
40 })
41 app.use(app.router);
42
43 // development only
44 if ('development' == app.get('env')) {
45   app.use(express.errorHandler());
46 }
47 app.param('task_id', function(req, res, next, taskId) {
48   req.db.tasks.findById(taskId, function(error, task){
49     if (error) return next(error);
50     if (!task) return next(new Error('Task is not found.'));
51     req.task = task;
52     return next();
53   });
54 });
55
56 app.get('/', routes.index);
57 app.get('/tasks', tasks.list);
58 app.post('/tasks', tasks.markAllCompleted)
59 app.post('/tasks', tasks.add);
60 app.post('/tasks/:task_id', tasks.markCompleted);
61 app.del('/tasks/:task_id', tasks.del);
62 app.get('/tasks/completed', tasks.completed);
63
64 app.all('*', function(req, res){
65   res.send(404);
66 })
67 http.createServer(app).listen(app.get('port'), function(){
68   console.log('Express server listening on port ' + app.get('port'));
69 });

2.5 Routes

There are only two files in routes folder. One of them serves the home page (e.g., http://localhost:3000/) and is straightforward:

1 /*
2  * GET home page.
3  */
4
5 exports.index = function(req, res){
6   res.render('index', { title: 'Express.js Todo App' });
7 };

The remaining logic that deals with tasks itself has been placed in the todo-express/routes/tasks.js. Let’s break it down a bit.

We start by exporting list() request handler that gives us list of incomplete tasks:

1 exports.list = function(req, res, next){

To do so, we perform a database search with completed=false query:

1   req.db.tasks.find({
2     completed: false
3   }).toArray(function(error, tasks){

In the callback, we need to check for any errors: javascript if (error) return next(error);

Since we use toArray(), we can send the date directly to the template:

1     res.render('tasks', {
2       title: 'Todo List',
3       tasks: tasks || []
4     });
5   });
6 };

Adding a new task requires us to check for the name parameter:

1 exports.add = function(req, res, next){
2   if (!req.body || !req.body.name)
3     return next(new Error('No data provided.'));

Thanks to our middleware, we already have a database collection in the req object, and the default value for the task is incomplete (completed: false):

1   req.db.tasks.save({
2     name: req.body.name,
3     completed: false
4   }, function(error, task){

Again, it’s important to check for errors and propagate them with Express.js next() function:

1     if (error) return next(error);
2     if (!task) return next(new Error('Failed to save.'));

The logging is optional; however, it’s useful for learning and debugging:

1     console.info('Added %s with id=%s', task.name, task._id);

Lastly, we redirect back to the Todo List page when the saving operation is finished successfully:

1     res.redirect('/tasks');
2   })
3 };

This method marks all incomplete tasks as complete:

1 exports.markAllCompleted = function(req, res, next) {

Because we had to re-use POST route and since it’s a good illustration of flow control, we check for the all_done parameter to decide if this request comes from the all done button or the add button:

1   if (!req.body.all_done
2     || req.body.all_done !== 'true')
3     return next();

If the execution come this far, we perform db query with multi: true: javascript req.db.tasks.update({ completed: false }, {$set: { completed: true }}, {multi: true}, function(error, count){

Significant error handling, logging and redirection back to Todo List page:

1     if (error) return next(error);
2     console.info('Marked %s task(s) completed.', count);
3     res.redirect('/tasks');
4   })
5 };

The Completed route is akin to Todo List except for the completed flag value (true in this case):

 1 exports.completed = function(req, res, next) {
 2   req.db.tasks.find({
 3     completed: true
 4   }).toArray(function(error, tasks) {
 5     res.render('tasks_completed', {
 6       title: 'Completed',
 7       tasks: tasks || []
 8     });
 9   });
10 };

This is the route that takes care of marking a single task as done. We use updateById but the same thing can be accomplished with a plain update method from Mongoskin/MongoDB API. The trick with completed: req.body.completed === 'true is needed because the incoming value is a string and not a boolean.

1 exports.markCompleted = function(req, res, next) {
2   if (!req.body.completed)
3     return next(new Error('Param is missing'));
4   req.db.tasks.updateById(req.task._id, {
5     $set: {completed: req.body.completed === 'true'}},
6     function(error, count) {

Once more, we perform error and results check (update() and updateById() don’t return object, but the count of affected documents instead):

 1       if (error) return next(error);
 2       if (count !==1)
 3         return next(new Error('Something went wrong.'));
 4       console.info('Marked task %s with id=%s completed.',
 5         req.task.name,
 6         req.task._id);
 7       res.redirect('/tasks');
 8     }
 9   )
10 }

Delete is the single route called by an AJAX request. However, there’s nothing special about its implementation. The only difference is that we don’t redirect, but send status 200 back.

Just for your information, the remove() method can be used instead of removeById().

 1 exports.del = function(req, res, next) {
 2   req.db.tasks.removeById(req.task._id, function(error, count) {
 3     if (error) return next(error);
 4     if (count !==1) return next(new Error('Something went wrong.'));
 5     console.info('Deleted task %s with id=%s completed.',
 6       req.task.name,
 7       req.task._id);
 8     res.send(200);
 9   });
10 }

For your convenience, here’s the full content of the todo-express/routes/tasks.js file:

 1 /*
 2  * GET users listing.
 3  */
 4
 5 exports.list = function(req, res, next){
 6   req.db.tasks.find({completed: false}).toArray(function(error, tasks){
 7     if (error) return next(error);
 8     res.render('tasks', {
 9       title: 'Todo List',
10       tasks: tasks || []
11     });
12   });
13 };
14
15 exports.add = function(req, res, next){
16   if (!req.body || !req.body.name) return next(new Error('No data provided.'));
17   req.db.tasks.save({
18     name: req.body.name,
19     completed: false
20   }, function(error, task){
21     if (error) return next(error);
22     if (!task) return next(new Error('Failed to save.'));
23     console.info('Added %s with id=%s', task.name, task._id);
24     res.redirect('/tasks');
25   })
26 };
27
28 exports.markAllCompleted = function(req, res, next) {
29   if (!req.body.all_done || req.body.all_done !== 'true') return next();
30   req.db.tasks.update({
31     completed: false
32   }, {$set: {
33     completed: true
34   }}, {multi: true}, function(error, count){
35     if (error) return next(error);
36     console.info('Marked %s task(s) completed.', count);
37     res.redirect('/tasks');
38   })
39 };
40
41 exports.completed = function(req, res, next) {
42   req.db.tasks.find({completed: true}).toArray(function(error, tasks) {
43     res.render('tasks_completed', {
44       title: 'Completed',
45       tasks: tasks || []
46     });
47   });
48 };
49
50 exports.markCompleted = function(req, res, next) {
51   if (!req.body.completed) return next(new Error('Param is missing'));
52   req.db.tasks.updateById(req.task._id, {$set: {completed: req.body.completed ===\
53  'true'}}, function(error, count) {
54     if (error) return next(error);
55     if (count !==1) return next(new Error('Something went wrong.'));
56     console.info('Marked task %s with id=%s completed.', req.task.name, req.task.\
57 _id);
58     res.redirect('/tasks');
59   })
60 };
61
62 exports.del = function(req, res, next) {
63   req.db.tasks.removeById(req.task._id, function(error, count) {
64     if (error) return next(error);
65     if (count !==1) return next(new Error('Something went wrong.'));
66     console.info('Deleted task %s with id=%s completed.', req.task.name, req.task\
67 ._id);
68     res.send(200);
69   });
70 };

2.6 Jades

In the Todo app, we use four templates:

• layout.jade: the skeleton of HTML pages that is used on all pages
• index.jade: home page
• tasks.jade: Todo List page
• tasks_completed.jade: Completed page

Let’s go through each file starting with layout.jade. It starts with doctype, html and head types:

1 doctype 5
2 html
3   head

We should have appname variable set:

1     title= title + ' | ' + appname

Next we include *.css files but underneath, Express.js will serve its contents from LESS files:

1     link(rel="stylesheet", href="/stylesheets/style.css")
2     link(rel="stylesheet", href="/bootstrap/bootstrap.css")
3     link(rel="stylesheet", href="/stylesheets/main.css")

The body with Twitter Bootstrap structure consist of .container and .navbar. To read more about those and other classes, go to getbootstrap.com/css/:

 1   body
 2     .container
 3       .navbar.navbar-default
 4         .container
 5           .navbar-header
 6             a.navbar-brand(href='/')= appname
 7       .alert.alert-dismissable
 8       h1= title
 9       p Welcome to Express.js Todo app by 
10         a(href='http://twitter.com/azat_co') @azat_co
11         |. Please enjoy.

This is the place where other jades (like tasks.jade) will be imported:

1       block content

The last lines include front-end JavaScript files:

1   script(src='/javascripts/jquery.js', type="text/javascript")
2   script(src='/javascripts/main.js', type="text/javascript")

The full layout.jade file:

 1 doctype 5
 2 html
 3   head
 4     title= title + ' | ' + appname
 5     link(rel="stylesheet", href="/stylesheets/style.css")
 6     link(rel="stylesheet", href="/bootstrap/bootstrap.css")
 7     link(rel="stylesheet", href="/stylesheets/main.css")
 8
 9   body
10     .container
11       .navbar.navbar-default
12         .container
13           .navbar-header
14             a.navbar-brand(href='/')= appname
15       .alert.alert-dismissable
16       h1= title
17       p Welcome to Express.js Todo app by 
18         a(href='http://twitter.com/azat_co') @azat_co
19         |. Please enjoy.
20       block content
21   script(src='/javascripts/jquery.js', type="text/javascript")
22   script(src='/javascripts/main.js', type="text/javascript")

The index.jade file is our home page and it’s quite vanilla. The most interesting thing it had is the nav-pills menu:

 1 extends layout
 2
 3 block content
 4   .menu
 5     h2 Menu
 6     ul.nav.nav-pills
 7       li.active
 8         a(href="/tasks") Home
 9       li
10         a(href="/tasks") Todo List
11       li
12         a(href="/tasks") Completed
13   .home
14     p This is an example of classical (no front-end JavaScript frameworks) web ap\
15 plication built with Express.js 3.3.5 for
16       a(href="http://expressjsguide.com") Express.js Guide
17       |.
18     p The full source code is available at
19       a(href='http://github.com/azat-co/todo-express') github.com/azat-co/todo-ex\
20 press
21       |.

The tasks.jade uses extends layout:

1 extends layout
2
3 block content

Then goes our main page specific content:

 1   .menu
 2     h2 Menu
 3     ul.nav.nav-pills
 4       li
 5         a(href='/') Home
 6       li.active
 7         a(href='/tasks') Todo List
 8       li
 9         a(href="/tasks/completed") Completed
10   h1= title

The div with list class will hold the Todo List:

1   .list
2     .item.add-task

The form to mark all items as done has CSRF token in a hidden field and uses POST method pointed to /tasks:

1       div.action
2         form(action='/tasks', method='post')
3           input(type='hidden', value='true', name='all_done')
4           input(type='hidden', value=locals._csrf, name='_csrf')
5           input(type='submit', class='btn btn-success btn-xs', value='all done')

Similar CSRF enabled form is for new task creation:

1       form(action="/tasks", method='post')
2         input(type='hidden', value=locals._csrf, name='_csrf')
3         div.name
4           input(type="text", name="name", placeholder='Add a new task')
5         div.delete
6           input.btn.btn-primary.btn-sm(type="submit", value='add')

When we start the app for the first time (or clean the database), there are no tasks:

1     if (tasks.length === 0)
2       | No tasks.

Jade supports iterations with each command:

1     each task, index in tasks
2       .item
3         div.action

This form submits data to individual task route:

1           form(action='/tasks/#{task._id}', method='post')
2             input(type='hidden', value=task._id.toString(), name='id')
3             input(type='hidden', value='true', name='completed')
4             input(type='hidden', value=locals._csrf, name='_csrf')
5             input(type='submit', class='btn btn-success btn-xs task-done', value=\
6 'done')

The index variable is used to display order in the list of tasks:

1         div.num
2           span=index+1
3             |. 
4         div.name
5           span.name=task.name
6           //- no support for DELETE method in forms
7           //- http://amundsen.com/examples/put-delete-forms/
8           //- so do XHR request instead from public/javascripts/main.js

The delete button doesn’t have anything fancy attached to it, because events are attached to these buttons from main.js front-end JavaScript file:

1         div.delete
2           a(class='btn btn-danger btn-xs task-delete', data-task-id=task._id.toSt\
3 ring(), data-csrf=locals._csrf) delete

The full source code of tasks.jade:

 1 extends layout
 2
 3 block content
 4
 5   .menu
 6     h2 Menu
 7     ul.nav.nav-pills
 8       li
 9         a(href='/') Home
10       li.active
11         a(href='/tasks') Todo List
12       li
13         a(href="/tasks/completed") Completed
14   h1= title
15
16   .list
17     .item.add-task
18       div.action
19         form(action='/tasks', method='post')
20           input(type='hidden', value='true', name='all_done')
21           input(type='hidden', value=locals._csrf, name='_csrf')
22           input(type='submit', class='btn btn-success btn-xs', value='all done')
23       form(action="/tasks", method='post')
24         input(type='hidden', value=locals._csrf, name='_csrf')
25         div.name
26           input(type="text", name="name", placeholder='Add a new task')
27         div.delete
28           input.btn.btn-primary.btn-sm(type="submit", value='add')
29     if (tasks.length === 0)
30       | No tasks.
31     each task, index in tasks
32       .item
33         div.action
34           form(action='/tasks/#{task._id}', method='post')
35             input(type='hidden', value=task._id.toString(), name='id')
36             input(type='hidden', value='true', name='completed')
37             input(type='hidden', value=locals._csrf, name='_csrf')
38             input(type='submit', class='btn btn-success btn-xs task-done', value=\
39 'done')
40         div.num
41           span=index+1
42             |. 
43         div.name
44           span.name=task.name
45           //- no support for DELETE method in forms
46           //- http://amundsen.com/examples/put-delete-forms/
47           //- so do XHR request instead from public/javascripts/main.js
48         div.delete
49           a(class='btn btn-danger btn-xs task-delete', data-task-id=task._id.toSt\
50 ring(), data-csrf=locals._csrf) delete

Last but not least, comes tasks_completed.jade which is just a striped down version of tasks.jade file:

 1 extends layout
 2
 3 block content
 4
 5   .menu
 6     h2 Menu
 7     ul.nav.nav-pills
 8       li
 9         a(href='/') Home
10       li
11         a(href='/tasks') Todo List
12       li.active
13         a(href="/tasks/completed") Completed
14
15   h1= title
16
17   .list
18     if (tasks.length === 0)
19       | No tasks.
20     each task, index in tasks
21       .item
22         div.num
23           span=index+1
24             |. 
25         div.name.completed-task
26           span.name=task.name

2.7 LESS

As we’ve mentioned before, after applying proper middleware in app.js files, we can put *.less files anywhere under public folder. Express.js works by accepting request for some .css file and tries to match corresponding file by name. Therefore, we include *.css files in our jades.

Here is the content of the todo-express/public/stylesheets/main.less file:

 1 * {
 2   font-size:20px;
 3 }
 4 .btn {
 5   // margin-left: 20px;
 6   // margin-right: 20px;
 7 }
 8 .num {
 9   // margin-right: 3px;
10 }
11 .item {
12   height: 44px;
13   width: 100%;
14   clear: both;
15   .name {
16     width: 300px;
17   }
18   .action {
19     width: 100px;
20   }
21   .delete {
22     width: 100px
23   }
24   div {
25     float:left;
26   }
27 }
28 .home {
29   margin-top: 40px;
30 }
31 .name.completed-task {
32   text-decoration: line-through;
33 }

2.8 Conclusion

The Todo app is considered classical because it doesn’t rely on any front-end framework. This was done intentionally to show how easy it is to use Express.js for such tasks. In modern day development, people often leverage some sort of REST API server architecture with front-end client built with Backbone.js, Angular, Ember or something else. Next examples dive into details about how to write such servers.

3 REST API

This tutorial will walk you through writing test using the Mocha and Super Agent libraries and then use them in a test-driven development manner to build a Node.js free JSON REST API server utilizing Express.js framework and Mongoskin library for MongoDB.

Example All the source code is in the github.com/azat-co/rest-api-express for your convenience.

In this REST API server, we’ll perform create, update, remove and delete (CRUD) operations and harness Express.js middleware concept with app.param() and app.use() methods.

3.1 Test Coverage

Before anything else, let’s write functional tests that make HTTP requests to our soon-to-be-created REST API server. If you know how to use Mocha or just want to jump straight to the Express.js app implementation, feel free to do so. You can use CURL terminal commands for testing too.

Assuming we already have Node.js, NPM and MongoDB installed, let’s create a new folder (or if you wrote the tests, use that folder):

1 mkdir rest-api
2 cd rest-api

We’ll use Mocha, Expect.js and Super Agent libraries. To install them, run these commands from the project folder:

1 $ npm install -g mocha
2 $ npm install expect.js
3 $ npm install superagent

Tip Installing Mocha locally will give us the ability to use different versions at the same time. To run tests, simply point to ./node_modules/mocha/bin/mocha. A better alternative would be to use Makefile as outlined in the HackHall example.

Now let’s create express.test.js file in the same folder which will have six suites:

• creating a new object
• retrieving an object by its ID
• retrieving the whole collection
• updating an object by its ID
• checking an updated object by its ID
• removing an object by its ID

HTTP requests are just a breeze with Super Agent’s chained functions which we’ll put inside of each test suite.

Here is the full source code for the rest-api-express/express.test.js file:

 1 var superagent = require('superagent')
 2 var expect = require('expect.js')
 3
 4 describe('express rest api server', function(){
 5   var id
 6
 7   it('post object', function(done){
 8     superagent.post('http://localhost:3000/collections/test')
 9       .send({ name: 'John'
10         , email: 'john@rpjs.co'
11       })
12       .end(function(e,res){
13         // console.log(res.body)
14         expect(e).to.eql(null)
15         expect(res.body.length).to.eql(1)
16         expect(res.body[0]._id.length).to.eql(24)
17         id = res.body[0]._id
18         done()
19       })
20   })
21
22   it('retrieves an object', function(done){
23     superagent.get('http://localhost:3000/collections/test/'+id)
24       .end(function(e, res){
25         // console.log(res.body)
26         expect(e).to.eql(null)
27         expect(typeof res.body).to.eql('object')
28         expect(res.body._id.length).to.eql(24)
29         expect(res.body._id).to.eql(id)
30         done()
31       })
32   })
33
34   it('retrieves a collection', function(done){
35     superagent.get('http://localhost:3000/collections/test')
36       .end(function(e, res){
37         // console.log(res.body)
38         expect(e).to.eql(null)
39         expect(res.body.length).to.be.above(0)
40         expect(res.body.map(function (item){return item._id})).to.contain(id)
41         done()
42       })
43   })
44
45   it('updates an object', function(done){
46     superagent.put('http://localhost:3000/collections/test/'+id)
47       .send({name: 'Peter'
48         , email: 'peter@yahoo.com'})
49       .end(function(e, res){
50         // console.log(res.body)
51         expect(e).to.eql(null)
52         expect(typeof res.body).to.eql('object')
53         expect(res.body.msg).to.eql('success')
54         done()
55       })
56   })
57
58   it('checks an updated object', function(done){
59     superagent.get('http://localhost:3000/collections/test/'+id)
60       .end(function(e, res){
61         // console.log(res.body)
62         expect(e).to.eql(null)
63         expect(typeof res.body).to.eql('object')
64         expect(res.body._id.length).to.eql(24)
65         expect(res.body._id).to.eql(id)
66         expect(res.body.name).to.eql('Peter')
67         done()
68       })
69   })
70   it('removes an object', function(done){
71     superagent.del('http://localhost:3000/collections/test/'+id)
72       .end(function(e, res){
73         // console.log(res.body)
74         expect(e).to.eql(null)
75         expect(typeof res.body).to.eql('object')
76         expect(res.body.msg).to.eql('success')
77         done()
78       })
79   })
80 })

To run the tests, we can use the $ mocha express.test.js command.

3.2 Dependencies

In this tutorial, we’ll utilize Mongoskin, a MongoDB library which is a better alternative to the plain good old native MongoDB driver for Node.js. In addition, Mongoskin is more light-weight than Mongoose and schema-less. For more insight, please check out Mongoskin comparison blurb.

Express.js is a wrapper for the core Node.js HTTP module objects. The Express.js framework is build on top of Connect middleware and provided tons of convenience. Some people compare the framework to Ruby’s Sinatra in terms of how it’s non-opinionated and configurable.

If you’ve create a rest-api folder in the previous section Test Coverage, simply run these commands to install modules for the application:

1 npm install express
2 npm install mongoskin

3.3 Implementation

First things first, so let’s define our dependencies:

1 var express = require('express')
2   , mongoskin = require('mongoskin')

After version 3.x, Express streamlines the instantiation of its app instance, in a way that this line will give us a server object:

1 var app = express()

To extract params from the body of the requests, we’ll use bodyParser() middleware which looks more like a configuration statement:

1 app.use(express.bodyParser())

Middleware (in this and other forms) is a powerful and convenient pattern in Express.js and Connect to organize and re-use code.

As with the bodyParser() method that saves us from the hurdles of parsing a body object of HTTP request, Mongoskin makes it possible to connect to the MongoDB database in one effortless line of code:

1 var db = mongoskin.db('localhost:27017/test', {safe:true});

Note If you wish to connect to a remote database, e.g., MongoHQ instance, substitute the string with your username, password, host and port values. Here is the format of the URI string: mongodb://[username:password@] host1[:port1][,host2[:port2],... [,hostN[:portN]]] [/[database][?options]]

The app.param() method is another Express.js middleware. It basically says “do something every time there is this value in the URL pattern of the request handler.” In our case, we select a particular collection when request pattern contains a string collectionName prefixed with a colon (you’ll see it later in the routes):

1 app.param('collectionName', function(req, res, next, collectionName){
2   req.collection = db.collection(collectionName)
3   return next()
4 })

Merely, to be user-friendly, let’s put a root route with a message:

1 app.get('/', function(req, res, next) {
2   res.send('please select a collection, e.g., /collections/messages')
3 })

Now the real work begins, here is how we retrieve a list of items sorted by _id and which has a limit of 10:

1 app.get('/collections/:collectionName', function(req, res, next) {
2   req.collection.find({},{
3     limit:10, sort: [['_id',-1]]
4   }).toArray(function(e, results){
5     if (e) return next(e)
6     res.send(results)
7   })
8 })

Have you noticed a :collectionName string in the URL pattern parameter? This and the previous app.param() middleware is what gives us the req.collection object which points to a specified collection in our database.

The object creating endpoint is slightly easier to grasp since we just pass the whole payload to the MongoDB (method a.k.a. free JSON REST API):

1 app.post('/collections/:collectionName', function(req, res, next) {
2   req.collection.insert(req.body, {}, function(e, results){
3     if (e) return next(e)
4     res.send(results)
5   })
6 })

Single object retrieval functions are faster than find(), but they use different interface (they return object directly instead of a cursor), so please be aware of that. In addition, we’re extracting the ID from :id part of the path with req.params.id Express.js magic:

1 app.get('/collections/:collectionName/:id', function(req, res, next) {
2   req.collection.findOne({
3     _id: req.collection.id(req.params.id)
4   }, function(e, result){
5     if (e) return next(e)
6     res.send(result)
7   })
8 })

PUT request handler gets more interesting because update() doesn’t return the augmented object; instead, it returns a count of affected objects.

Also {$set:req.body} is a special MongoDB operator (operators tend to start with a dollar sign) that sets values.

The second ` {safe:true, multi:false}` parameter is an object with options that tell MongoDB to wait for the execution before running the callback function and to process only one (first) item.

 1 app.put('/collections/:collectionName/:id', function(req, res, next) {
 2   req.collection.update({
 3       _id: req.collection.id(req.params.id)
 4     }, {$set:req.body}, {safe:true, multi:false},
 5     function(e, result){
 6       if (e) return next(e)
 7       res.send((result===1)?{msg:'success'}:{msg:'error'})
 8     }
 9   );
10 })

Finally, the DELETE method which also outputs a custom JSON message:

 1 app.del('/collections/:collectionName/:id', function(req, res, next) {
 2   req.collection.remove({
 3       _id: req.collection.id(req.params.id)
 4     },
 5     function(e, result){
 6       if (e) return next(e)
 7       res.send((result===1)?{msg:'success'}:{msg:'error'})
 8     }
 9   );
10 })

Note: The delete is an operator in JavaScript, so Express.js uses app.del instead.

The last line that actually starts the server on port 3000 in this case:

1 app.listen(3000)

Just in case something is not working quite well, here is the full code of rest-api-express/express.js file :

 1 var express = require('express')
 2   , mongoskin = require('mongoskin')
 3
 4 var app = express()
 5 app.use(express.bodyParser())
 6
 7 var db = mongoskin.db('localhost:27017/test', {safe:true});
 8
 9 app.param('collectionName', function(req, res, next, collectionName){
10   req.collection = db.collection(collectionName)
11   return next()
12 })
13
14 app.get('/', function(req, res, next) {
15   res.send('please select a collection, e.g., /collections/messages')
16 })
17
18 app.get('/collections/:collectionName', function(req, res, next) {
19   req.collection.find({},{limit:10, sort: [['_id',-1]]}).toArray(function(e, resu\
20 lts){
21     if (e) return next(e)
22     res.send(results)
23   })
24 })
25
26 app.post('/collections/:collectionName', function(req, res, next) {
27   req.collection.insert(req.body, {}, function(e, results){
28     if (e) return next(e)
29     res.send(results)
30   })
31 })
32
33
34 app.get('/collections/:collectionName/:id', function(req, res, next) {
35   req.collection.findOne({_id: req.collection.id(req.params.id)}, function(e, res\
36 ult){
37     if (e) return next(e)
38     res.send(result)
39   })
40 })
41 app.put('/collections/:collectionName/:id', function(req, res, next) {
42   req.collection.update({_id: req.collection.id(req.params.id)}, {$set:req.body},\
43  {safe:true, multi:false}, function(e, result){
44     if (e) return next(e)
45     res.send((result===1)?{msg:'success'}:{msg:'error'})
46   })
47 })
48 app.del('/collections/:collectionName/:id', function(req, res, next) {
49   req.collection.remove({_id: req.collection.id(req.params.id)}, function(e, resu\
50 lt){
51     if (e) return next(e)
52     res.send((result===1)?{msg:'success'}:{msg:'error'})
53   })
54 })
55
56
57
58 app.listen(3000)

Exit your editor and run this command in your terminal:

1 $ node express.js

And in a different window (without closing the first one):

1 $ mocha express.test.js

If you really don’t like Mocha and/or BDD, CURL is always there for you. :-)

For example, CURL data to make a POST request:

1 $ curl -d "" http://localhost:3000

GET requests also work in the browser, for example http://localhost:3000/test.

In this tutorial, our tests are longer than the app code itself so abandoning test-driven development might be tempting, but believe me, the good habits of TDD will save you hours and hours during any serious development when the complexity of the application you are working on is big.

3.4 Conclusion

The Express.js and Mongoskin libraries are great when you need to build a simple REST API server in a few lines of code. Later, if you need to expand the libraries, they also provide a way to configure and organize your code.

NoSQL databases like MongoDB are good at free-REST APIs where we don’t have to define schemas and can throw any data and it’ll be saved.

The full code for both test and app files: https://gist.github.com/azat-co/6075685.

If you would like to learn more about Express.js and other JavaScript libraries, take a look at the series Intro to Express.js tutorials.

Note: *In this example, I’m using semi-colon less style. Semi-colons in JavaScript are absolutely optional except in two cases: in the for loop and before expressions/statements that start with parenthesis (e.g., Immediately-Invoked Function Expression).

4 HackHall

Example The HackHall source code is in the public GitHub repository.

The live demo is accessible at hackhall.com either with AngelList or pre-filled email (1@1.com) and password (1).

4.1 What is HackHall

HackHall (ex-Accelerator.IO) is an open-source invite-only social network and collaboration tool for hackers, hipsters, entrepreneurs and pirates (just kidding). HackHall is akin to Reddit, plus Hacker News, plus Facebook Groups with curation.

The HackHall project is in its early stage and roughly a beta. We plan to extend the code base in the future and bring in more people to share skills, wisdom and passion for programming.

In this chapter, we’ll cover the 1.0 release which has:

• OAuth 1.0 with oauth modules and AngelList API
• Email and password authentication
• Mongoose models and schemas
• Express.js structure with routes in modules
• JSON REST API
• Express.js error handling
• Front-end client Backbone.js app (for more info on Backbone.js, download/read online our Rapid Prototyping with JS tutorials)
• Environmental variables with Foreman’s .env
• TDD with Mocha
• Basic Makefile setup

4.2 Running HackHall

To get the source code, you can navigate to hackhall folder or clone from GitHub:

1 $ git clone git@github.com:azat-co/hackhall
2 $ git checkout 1.0
3 $ npm install

If you plan to test an AngelList (optional), HackHall is using Heroku and Foreman setup for AngelList API keys storing them in environmental variables, so we need to add .evn file like this (below are fake values):

1 ANGELLIST_CLIENT_ID=254C0335-5F9A-4607-87C0
2 ANGELLIST_CLIENT_SECRET=99F5C1AC-C5F7-44E6-81A1-8DF4FC42B8D9

The keys are obtainable at angel.co/api after someone creates and registers his/her AngelList app.

Download and install MongoDB if you don’t have it already. The databases and third-party libraries are out of scope of this book. However, you can find enough materials online and in Rapid Prototyping with JS.

To start MongoDB server, open a new terminal window and run:

1 $ mongod

Go back to the project folder and run:

1 $ foreman start

After MongoDB is running on localhost with default port 27017, it’s possible to seed the database hackhall with default admin user by running seed.js mongo script:

1 $ mongo localhost:27017/hackhall seed.js

Feel free to modify seed.js to your liking (beware that it erases all previous data!):

 1 db.dropDatabase();
 2 var seedUser ={
 3   firstName:'Azat',
 4   lastName:"Mardanov",
 5   displayName:"Azat Mardanov",
 6   password:'1',
 7   email:'1@1.com',
 8   role:'admin',
 9   approved: true,
10   admin: true
11 };
12 db.users.save(seedUser);

If you open your browser at http://localhost:5000, you should see the login screen.

The HackHall login page running locally.

Enter username and password to get in (the ones from the seed.js file).

The HackHall login page with seed data.

After successful authentication, users are redirected to the Posts page:

The HackHall Posts page.

where they can create a post (e.g., a question):

The HackHall Posts page.

Save the post:

The HackHall Posts page with a saved post.

Like posts:

The HackHall Posts page with a liked post.

Visit other users’ profiles:

The HackHall People page.

If they have admin rights, users can approve applicants:

The HackHall People page with admin rights.

and manage their account on Profile page:

The HackHall Profile page.

4.3 Structure

Here are what each of the folders and files have:

• /api: app-shared routes
• /models: Mongoose models
• /public: Backbone app, static files like front-end JavaScript, CSS, HTML
• /routes: REST API routes
• /tests: Mocha tests
• .gitignore: list of files that git should ignore
• Makefile: make file to run tests
• Procfile: Cedar stack file needed for Heroku deployment
• package.json: NPM dependencies and HackHall meta data
• readme.md: description
• server.js: main HackHall server file

Content of the HackHall base folder.

4.4 Express.js App

Let’s jump straight to the server.js file and learn how it’s implemented. Firstly, we declare dependencies:

1 var express = require('express'),
2   routes = require('./routes'),
3   http = require('http'),
4   util = require('util'),
5   oauth = require('oauth'),
6   querystring = require('querystring');

Then, we initialize app and configure middlewares. The process.env.PORT is populated by Heroku, and in case of a local setup fallsback on 3000.

1 var app = express();
2 app.configure(function(){
3   app.set('port', process.env.PORT || 3000  );
4   app.use(express.favicon());
5   app.use(express.logger('dev'));
6   app.use(express.bodyParser());
7   app.use(express.methodOverride());

The values passed to cookieParser and session middlewares are needed for authentication. Obviously, session secrets are supposed to be private:

1   app.use(express.cookieParser('asd;lfkajs;ldfkj'));
2   app.use(express.session({
3     secret: '<h1>WHEEYEEE</h1>',
4     key: 'sid',
5     cookie: {
6       secret: true,
7       expires: false
8     }
9   }));

This is how we serve our front-end client Backbone.js app and other static files like CSS:

1   app.use(express.static(__dirname + '/public'));
2   app.use(app.router);
3 });

Error handling is broken down into three functions with clientErrorHandler dedicated to AJAX/XHR requests from the Backbone.js app (responds with JSON):

 1 app.configure(function() {
 2   app.use(logErrors);
 3   app.use(clientErrorHandler);
 4   app.use(errorHandler);
 5 });
 6
 7 function logErrors(err, req, res, next) {
 8   console.error('logErrors', err.toString());
 9   next(err);
10 }
11
12 function clientErrorHandler(err, req, res, next) {
13   console.error('clientErrors ', err.toString());
14   res.send(500, { error: err.toString()});
15   if (req.xhr) {
16     console.error(err);
17     res.send(500, { error: err.toString()});
18   } else {
19     next(err);
20   }
21 }
22
23 function errorHandler(err, req, res, next) {
24   console.error('lastErrors ', err.toString());
25   res.send(500, {error: err.toString()});
26 }

In the same way that we determine process.env.PORT and fallback on local setup value 3000, we do a similar thing with MongoDB connection string:

1 var dbUrl = process.env.MONGOHQ_URL
2   || 'mongodb://@127.0.0.1:27017/hackhall';
3 var mongoose = require('mongoose');
4 var connection = mongoose.createConnection(dbUrl);
5 connection.on('error', console.error.bind(console,
6   'connection error:'));

Sometime it’s a good idea to log on the connection open event:

1 connection.once('open', function () {
2   console.info('connected to database')
3 });

The Mongoose models live in the models folder:

1 var models = require('./models');

This middleware will provide access to two collections within our routes methods:

1 function db (req, res, next) {
2   req.db = {
3     User: connection.model('User', models.User, 'users'),
4     Post: connection.model('Post', models.Post, 'posts')
5   };
6   return next();
7 }

Just a new name for imported auth functions:

1 checkUser = routes.main.checkUser;
2 checkAdmin = routes.main.checkAdmin;
3 checkApplicant = routes.main.checkApplicant;

AngelList OAuth routes:

1 app.get('/auth/angellist', routes.auth.angelList);
2 app.get('/auth/angellist/callback',
3   routes.auth.angelListCallback,
4   routes.auth.angelListLogin,
5   db,
6   routes.users.findOrAddUser);

Main application routes including api/profile return user session if user is logged in:

1 //MAIN
2 app.get('/api/profile', checkUser, db, routes.main.profile);
3 app.del('/api/profile', checkUser, db, routes.main.delProfile);
4 app.post('/api/login', db, routes.main.login);
5 app.post('/api/logout', routes.main.logout);

The POST requests for creating users and posts:

 1 //POSTS
 2 app.get('/api/posts', checkUser, db, routes.posts.getPosts);
 3 app.post('/api/posts', checkUser, db, routes.posts.add);
 4 app.get('/api/posts/:id', checkUser, db, routes.posts.getPost);
 5 app.put('/api/posts/:id', checkUser, db, routes.posts.updatePost);
 6 app.del('/api/posts/:id', checkUser, db, routes.posts.del);
 7
 8 //USERS
 9 app.get('/api/users', checkUser, db, routes.users.getUsers);
10 app.get('/api/users/:id', checkUser, db,routes.users.getUser);
11 app.post('/api/users', checkAdmin, db, routes.users.add);
12 app.put('/api/users/:id', checkAdmin, db, routes.users.update);
13 app.del('/api/users/:id', checkAdmin, db, routes.users.del);

These routes are for new members that haven’t been approved yet:

 1 //APPLICATION
 2 app.post('/api/application',
 3   checkAdmin,
 4   db,
 5   routes.application.add);
 6 app.put('/api/application',
 7   checkApplicant,
 8   db,
 9   routes.application.update);
10 app.get('/api/application',
11   checkApplicant,
12   db,
13   routes.application.get);

The catch all else route:

1 app.get('*', function(req, res){
2   res.send(404);
3 });

The require.main === module is a clever trick to determine if this file is being executed as a stand alone or as an imported module:

 1 http.createServer(app);
 2 if (require.main === module) {
 3   app.listen(app.get('port'), function(){
 4     console.info('Express server listening on port '
 5       + app.get('port'));
 6   });
 7 }
 8 else {
 9   console.info('Running app as a module')
10   exports.app = app;
11 }

The full source code for hackhall/server.js:

  1 var express = require('express'),
  2   routes = require('./routes'),
  3   http = require('http'),
  4   util = require('util'),
  5   oauth = require('oauth'),
  6   querystring = require('querystring');
  7
  8 var app = express();
  9 app.configure(function(){
 10   app.set('port', process.env.PORT || 3000  );
 11   app.use(express.favicon());
 12   app.use(express.logger('dev'));
 13   app.use(express.bodyParser());
 14   app.use(express.methodOverride());
 15   app.use(express.cookieParser('asd;lfkajs;ldfkj'));
 16   app.use(express.session({
 17     secret: '<h1>WHEEYEEE</h1>',
 18     key: 'sid',
 19     cookie: {
 20       secret: true,
 21       expires: false
 22     }
 23   }));
 24   // app.use(express.csrf());
 25   // app.use(function(req, res, next) {
 26     // res.locals.csrf = req.session._cstf;
 27     // return next();
 28   // });
 29   app.use(express.static(__dirname + '/public'));
 30   app.use(app.router);
 31 });
 32
 33 app.configure(function() {
 34   app.use(logErrors);
 35   app.use(clientErrorHandler);
 36   app.use(errorHandler);
 37 });
 38
 39 function logErrors(err, req, res, next) {
 40   console.error('logErrors', err.toString());
 41   next(err);
 42 }
 43
 44 function clientErrorHandler(err, req, res, next) {
 45   console.error('clientErrors ', err.toString());
 46   res.send(500, { error: err.toString()});
 47   if (req.xhr) {
 48     console.error(err);
 49     res.send(500, { error: err.toString()});
 50   } else {
 51     next(err);
 52   }
 53 }
 54
 55 function errorHandler(err, req, res, next) {
 56   console.error('lastErrors ', err.toString());
 57   res.send(500, {error: err.toString()});
 58 }
 59
 60 var dbUrl = process.env.MONGOHQ_URL || 'mongodb://@127.0.0.1:27017/hackhall';
 61 var mongoose = require('mongoose');
 62 var connection = mongoose.createConnection(dbUrl);
 63 connection.on('error', console.error.bind(console, 'connection error:'));
 64 connection.once('open', function () {
 65   console.info('connected to database')
 66 });
 67
 68 var models = require('./models');
 69 function db (req, res, next) {
 70   req.db = {
 71     User: connection.model('User', models.User, 'users'),
 72     Post: connection.model('Post', models.Post, 'posts')
 73   };
 74   return next();
 75 }
 76 checkUser = routes.main.checkUser;
 77 checkAdmin = routes.main.checkAdmin;
 78 checkApplicant = routes.main.checkApplicant;
 79
 80 app.get('/auth/angellist', routes.auth.angelList);
 81 app.get('/auth/angellist/callback',
 82   routes.auth.angelListCallback,
 83   routes.auth.angelListLogin,
 84   db,
 85   routes.users.findOrAddUser);
 86
 87 //MAIN
 88 app.get('/api/profile', checkUser, db, routes.main.profile);
 89 app.del('/api/profile', checkUser, db, routes.main.delProfile);
 90 app.post('/api/login', db, routes.main.login);
 91 app.post('/api/logout', routes.main.logout);
 92
 93 //POSTS
 94 app.get('/api/posts', checkUser, db, routes.posts.getPosts);
 95 app.post('/api/posts', checkUser, db, routes.posts.add);
 96 app.get('/api/posts/:id', checkUser, db, routes.posts.getPost);
 97 app.put('/api/posts/:id', checkUser, db, routes.posts.updatePost);
 98 app.del('/api/posts/:id', checkUser, db, routes.posts.del);
 99
100 //USERS
101 app.get('/api/users', checkUser, db, routes.users.getUsers);
102 app.get('/api/users/:id', checkUser, db,routes.users.getUser);
103 app.post('/api/users', checkAdmin, db, routes.users.add);
104 app.put('/api/users/:id', checkAdmin, db, routes.users.update);
105 app.del('/api/users/:id', checkAdmin, db, routes.users.del);
106
107 //APPLICATION 
108 app.post('/api/application', checkAdmin, db, routes.application.add);
109 app.put('/api/application', checkApplicant, db, routes.application.update);
110 app.get('/api/application', checkApplicant, db, routes.application.get);
111
112 app.get('*', function(req, res){
113   res.send(404);
114 });
115
116 http.createServer(app);
117 if (require.main === module) {
118   app.listen(app.get('port'), function(){
119     console.info('Express server listening on port ' + app.get('port'));
120   });
121 }
122 else {
123   console.info('Running app as a module')
124   exports.app = app;
125 }