Node Cookbook

By David Mark Clements

As part of Packt’s cookbook series, this book is packed with practical recipes that will get you working efficiently with Node from the start. Each chapter focuses on a different aspect of working with Node. If you have some knowledge of JavaScript and want to build fast, efficient, scalable client-server solutions, then Node Cookbook is for you. Experienced users of Node will improve their skills although even if you have not worked with Node before, these practical recipes will make it easy to get started.

• Language: English
• Publisher: Packt Publishing
• Release date: Jul 25, 2012
• ISBN: 9781849517195

Book preview

Table of Contents

Node Cookbook

Credits

About the Author

About the Reviewers

www.PacktPub.com

Support files, eBooks, discount offers and more

Why Subscribe?

Free Access for Packt account holders

Preface

What this book covers

What you need for this book

Who this book is for

Conventions

Reader feedback

Customer support

Downloading the example code

Errata

Piracy

Questions

1. Making a Web Server

Introduction

Setting up a router

Getting ready

How to do it...

How it works...

There's more...

Simple multilevel routing

Parsing the querystring

Routing modules

See also

Serving static files

Getting ready

How to do it...

How it works...

There's more...

The favicon gotcha

See also

Caching content in memory for immediate delivery

Getting ready

How to do it...

How it works...

There's more...

Reflecting content changes

See also

Optimizing performance with streaming

Getting ready

How to do it...

How it works...

There's more...

Protecting against process memory overruns

See also

Securing against filesystem hacking exploits

Getting ready

How to do it...

How it works...

There's more...

Whitelisting

Node-static

See also

2. Exploring the HTTP Object

Introduction

Processing POST data

Getting ready

How to do it...

How it works...

There's more...

Accessing POST data with connect.bodyParser

See also

Handling file uploads

Getting ready

How to do it...

How it works...

There's more...

Using formidable to accept all POST data

Preserving filenames with formidable

Uploading via PUT

See also

Using Node as an HTTP client

Getting ready

How to do it...

How it works...

There's more...

Sending POST requests

Multipart file upload as a client

See also

Implementing download throttling

Getting ready

How to do it...

How it works...

Enabling resumes from broken downloads

See also

3. Working with Data Serialization

Introduction

Converting an object to JSON and back again

Getting ready

How to do it...

How it works...

There's more...

Constructing JSONP responses

Security and JSONP

See also

Converting an object to XML and back again

Getting ready

How to do it...

How it works...

There's more...

Objects containing arrays and functions

Generating XML attributes

Text values alongside attribute declarations

See also

Browser-server transmission via AJAX

Getting ready

How to do it...

How it works...

There's more...

Sending serialized data from client to server

See also

Working with real data: fetching trending tweets

Getting ready

How to do it...

How it works...

Twitter API and the User-Agent header

There's more...

Cross referencing Google Hot Trends with Twitter tweets

See also

4. Interfacing with Databases

Introduction

Writing to a CSV file

Getting ready

How to do it...

How it works...

There's more...

Customizing the CSV elements

Reading a CSV file

Manipulating CSV as a stream

See also

Connecting and sending SQL to a MySQL server

Getting ready

How to do it...

How it works...

There's more...

Using and cleaning user input

Receiving results from the MySQL server

See also

Storing and retrieving data with MongoDB

Getting ready

How to do it...

How it works...

There's more...

Indexing and aggregation

Updating modifiers, sort, and limit

See also

Storing and retrieving data with Mongoskin

Getting ready

How to do it...

How it works...

There's more...

Collection binding

See also

Storing data to CouchDB with Cradle

Getting ready

How to do it...

How it works...

There's more...

Scaling CouchDB with BigCouch

See also

Retrieving data from CouchDB with Cradle

Getting ready

How to do it...

How it works...

There's more...

Creating an admin user

Locking all modifying operations to an admin user

Exposing the CouchDB HTTP interface to remote connections

See also

Accessing CouchDB changes stream with Cradle

Getting ready

How to do it...

How it works...

See also

Storing and retrieving data with Redis

Getting ready

How to do it...

How it works...

There's more...

Speeding up node Redis module

Overcoming network latency by pipelining commands

See also

Implementing PubSub with Redis

Getting ready

How to do it...

How it works...

There's more...

Redis authentication

Securing Redis from external connections

See also

5. Transcending AJAX: Using WebSockets

Introduction

Creating a WebSocket server

Getting ready

How to do it...

How it works...

There's more...

Supporting older Firefox browsers

Creating a node-based WebSocket client

See also

Seamless fallbacking with socket.io

Getting ready

How to do it...

How it works...

There's more...

Custom events

Namespaces

See also

Callbacks over socket.io transport

Getting ready

How to do it...

How it works...

There's more...

Shared functions with Nowjs

See also

Creating a real-time widget

Getting ready

How to do it...

How it works...

There's more...

Preparing for scalability

WebSockets as a development tool

See also

6. Accelerating Development with Express

Introduction

Generating Express scaffolding

Getting ready

How to do it...

How it works...

There's more...

Picking apart app.js

Looking into routes/index.js

See also

Defining and applying environments

Getting ready

How to do it...

How it works...

There's more...

Setting other environments

Changing NODE_ENV permanently

See also

Dynamic routing

Getting ready

How to do it...

How it works...

There's more...

Route validation

Optional routes

Asterisks wildcards

See also

Templating in Express

Getting ready

How to do it...

How it works...

There's more...

Using other template engines

EJS templates

Literal JavaScript in Jade

Jade partials

Express partials

Jade includes

layout.jade

See also

CSS engines with Express

Getting ready

How to do it...

How it works...

There's more...

Nested mixins and rest parameters

Playing with colors

Using the LESS engine

See also

Initializing and using a session

Getting ready

How to do it...

How it works...

There's more...

Custom middleware for site-wide session management

Flash messages

See also

Making an Express web app

Getting ready

How to do it...

Creating a database bridge

Configuring app.js files

Modifying the profiler app

Modifying the mounted login App

How it works...

App mounting

Data Flow

Route handling

Views

Mixins

Helpers

Styles

User flow

There's more...

Benchmarking

Using logger

See also

7. Implementing Security, Encryption, and Authentication

Introduction

Implementing Basic Authentication

Getting ready

How to do it...

How it works...

There's more...

Basic Authentication with Express

See also

Cryptographic password hashing

Getting ready

How to do it...

How it works...

There's more...

Uniquifying hashes with HMAC

Hardened hashing with PBKDF2

See also

Implementing Digest Authentication

Getting ready

How to do it...

How it works...

There's more...

Logging out of authenticated areas

See also

Setting up an HTTPS web server

Getting ready

How to do it...

How it works...

There's more...

HTTPS in Express

Securing Basic Authentication with SSL/TLS

See also

Preventing cross-site request forgery

Getting ready

How to do it...

How it works...

Cross-site scripting (XSS) circumvention

There's more...

Auto-securing the POST forms with the CSRF elements

Eliminating cross-site scripting (XSS) vulnerabilities

See also

8. Integrating Network Paradigms

Introduction

Sending email

Getting ready

How to do it...

How it works...

There's more...

Using sendmail as an alternative transport

HTML emails

Sending attachments

See also

Sending SMS

Getting ready

How to do it...

How it works...

There's more...

Using the processed event listener

Making an automated phone call

See also

Communicating with TCP

Getting ready

How to do it...

How it works...

There's more...

Port forwarding

Using pcap to watch TCP traffic

See also

Creating an SMTP server

Getting ready

How to do it...

How it works...

There's more...

Receiving emails from external SMTP servers

See also

Implementing a virtual hosting paradigm

Getting ready

How to do it...

How it works...

There's more...

Virtual hosting Express apps

Server Name Indication

See also

9. Writing Your Own Node Modules

Introduction

Creating a test-driven module API

Getting ready

How to do it...

How it works...

There's more...

Unit tests with should.js

See also

Writing a functional module mock-up

Getting ready

How to do it...

How it works...

There's more...

Writing a module use case example

See also

Refactoring from functional to prototypical

Getting ready

How to do it...

How it works...

There's more...

Adding the stat function to the initialized mp3dat object

Allowing for multiple instances

See also

Extending the module's API

Getting ready

How to do it...

How it works...

There's more...

Making the STDIN stream example

Making the PUT upload stream example

Merging stat and statStream

Integrating the EventEmitter

See also

Deploying a module to npm

Getting ready

How to do it...

How it works...

There's more...

npm link

.npmignore and npm version

See also

10. Taking It Live

Introduction

Deploying to a server environment

Getting ready

How to do it...

How it works...

There's more...

Using screen instead of nohup

Using authbind for privileged ports

Hosting multiple processes from port 80

See also

Automatic crash recovery

Getting ready

How to do it...

How it works...

There's more...

Detecting a respawn limit violation

Staying up with forever

See also

Continuous deployment

Getting ready

How to do it...

How it works...

There's more...

Building module dependencies on update

Writing a Node Git hook for integrated testing

See also

Hosting with a Platform as a Service provider

Getting ready

How to do it...

How it works...

There's more...

Assigning custom domains to Nodejitsu apps

Provisioning a database with jitsu

See also

Index

Node Cookbook

---

Node Cookbook

Copyright © 2012 Packt Publishing

All rights reserved. No part of this book may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, without the prior written permission of the publisher, except in the case of brief quotations embedded in critical articles or reviews.

Every effort has been made in the preparation of this book to ensure the accuracy of the information presented. However, the information contained in this book is sold without warranty, either express or implied. Neither the author, nor Packt Publishing, and its dealers and distributors will be held liable for any damages caused or alleged to be caused directly or indirectly by this book.

Packt Publishing has endeavored to provide trademark information about all of the companies and products mentioned in this book by the appropriate use of capitals. However, Packt Publishing cannot guarantee the accuracy of this information.

First published: July 2012

Production Reference: 1120712

Published by Packt Publishing Ltd.

Livery Place

35 Livery Street

Birmingham B3 2PB, UK..

ISBN 978-1-84951-718-8

www.packtpub.com

Cover Image by Asher Wishkerman (<[e-mail]>)

Credits

Author

David Mark Clements

Reviewers

Marc Harter

Joachim Larsen

Joshua Holbrook

Acquisition Editor

Sarah Cullington

Lead Technical Editor

Jonathan Titmus

Technical Editor

Manasi Poonthottam

Project Coordinator

Sai Gamare

Proofreader

Joel T. Johnson

Indexer

Rekha Nair

Graphics

Manu Joseph

Production Coordinator

Shantanu Zagade

Cover Work

Shantanu Zagade

About the Author

David Mark Clements is a web entrepreneur residing in Northern Ireland. From a very early age he was fascinated with programming and computers. He first learned BASIC on one of the many Atari’s he had accumulated by the age of nine. David learned JavaScript at the age of 12, moving into Linux administration and PHP as a teenager.

Now (as a twenty-something) he uses his technical skills in business, providing a variety of services ranging from tailored websites, to complete online strategies, to implementing custom warehouse logistics integrated with e-commerce solutions. Node has become a prominent member of his toolkit due to its versatility and powerful interfacing abilities.

When he’s not tinkering with computers, he’s working in a homeless hostel, playing bass with Living Rivers worship team, or spending time with the love of his life, Maxine, and her Husky-Spitz cross, Jessi.

Many thanks to the Node community, who have caused Node to grow as it has, and Node Google Group, which has been an immense source of information and inspiration.

I cannot conclude without acknowledging Jesus, who makes my life worthwhile and gave me strength to complete this book among all the other commitments, interruptions, and general life events.

About the Reviewers

Marc Harter is a Senior Web Developer for Applied Data Consultants in Eau Claire, WI. He is actively involved in the JavaScript community from the browser to the server side. He enjoys studying programming languages and paradigms, obsessively changing his theme and background on ArchLinux, and keeping up with the latest in web development. He has a beautiful wife and two beautiful children.

Joachim Larsen is interested in making stuff work. His appetite for learning new skills and applying them in new ways has seen him build total-solution systems for computer cafes; Win95 Active Desktop HTML frontend, Samba backend with save-game propagation, Windows Registry manipulation — including CD key pools for optimizing the use of purchased game inventory, networked-based CD image mounting, and more. Being a restless sort, he took to working offshore, building and installing boxes for getting the most out of satellite-based Internet connections while ensuring a hassle-free experience for the individual users.

Currently, his focus is on making the user experience faster, and making it faster to make more user experiences — generally, in the context of JavaScript-rich web applications.

He co-hosts the http://javascriptjabber.com/ podcast and dabbles in applying electronics to small-scale food production.

I would like to thank my parents for a very interesting childhood that continues to inspire and inform. I love you guys :)

Joshua Holbrook, who hails from the far North, earned a master’s degree in Mechanical Engineering at the University of Alaska Fairbanks, where he successfully defended his thesis on The measurement of anisotropic thermal conductivity in snow with needle probes. His original background is in MATLAB and Python, but he has since mastered Node.js. He has now moved to sunny California with his bird Korben to work in startups.

www.PacktPub.com

Support files, eBooks, discount offers and more

You might want to visit www.PacktPub.com for support files and downloads related to your book.

Did you know that Packt offers eBook versions of every book published, with PDF and ePub files available? You can upgrade to the eBook version at www.PacktPub.com and as a print book customer, you are entitled to a discount on the eBook copy. Get in touch with us at for more details.

At www.PacktPub.com, you can also read a collection of free technical articles, sign up for a range of free newsletters and receive exclusive discounts and offers on Packt books and eBooks.

http://PacktLib.PacktPub.com

Do you need instant solutions to your IT questions? PacktLib is Packt’s online digital book library. Here, you can access, read and search across Packt’s entire library of books.

Why Subscribe?

Fully searchable across every book published by Packt

Copy and paste, print and bookmark content

On demand and accessible via web browser

Free Access for Packt account holders

If you have an account with Packt at www.PacktPub.com, you can use this to access PacktLib today and view nine entirely free books. Simply use your login credentials for immediate access.

Preface

The principles of asynchronous event-driven programming are perfect for today's Web, where efficient real-time applications and scalability are at the forefront. Server-side JavaScript has been here since the nineties, but Node got it right. With a thriving community and interest from the Internet giants, it could be the PHP of tomorrow.

Node Cookbook shows you how to transfer your JavaScript skills to server-side programming. With simple examples and supporting code, Node Cookbook takes you through various server-side scenarios, often saving you time, effort, and trouble by demonstrating best practices and showing you how to avoid security mistakes.

Beginning with making your own web server, the practical recipes in this cookbook are designed to smoothly progress you to making full web applications, command-line applications, and Node modules. Node Cookbook takes you through interfacing with various database backends, such as MySQL, MongoDB, and Redis, working with web sockets, and interfacing with network protocols, such as SMTP. Additionally, there are recipes on handling streams of data, security implementations, writing, your own Node modules, and different ways to take your apps live.

What this book covers

Chapter 1, Making a Web Server, covers serving dynamic and static content, caching files in memory, streaming large files straight from disk over HTTP, and securing your web server.

Chapter 2, Exploring the HTTP Object, explains receiving and processing POST requests and file uploads, using Node as an HTTP client, and discusses how to throttle downloads.

Chapter 3, Working with Data Serialization, explains how to convert data from your apps into XML and JSON formats for sending to the browser or third-party APIs.

Chapter 4, Interfacing with Databases, covers how to implement persistent data stores with Redis, CouchDB, MongoDB, MySQL, or plain CSV files.

Chapter 5, Transcending AJAX: Using WebSockets, helps you to make real-time web apps with modern browser WebSocket technology, gracefully degrade to long-polling and other methods with Socket.io.

Chapter 6, Accelerating Development with Express, explains how to leverage the Express framework to achieve rapid web development. It also covers using template languages and CSS engines, such as LESS and Stylus.

Chapter 7, Implementing Security, Encryption, and Authentication, explains how to set up an SSL secured web server, use the crypto module to create strong password hashes, and protect your users from cross-site request forgery attacks.

Chapter 8, Integrating Network Paradigms, discusses sending emails and creating your own email server, sending SMS text messages, implementing virtual hosting, and doing fun and interesting things with raw TCP.

Chapter 9, Writing Your Own Node Modules explains how to create a test suite, write a solution, refactor, improve and extend, and then deploy your own Node module.

Chapter 10, Taking It Live discusses how to deploy your web apps to a live server, ensure your apps stay live with crash recovery techniques, implement a continuous deployment work flow, or alternatively simply use a Platform as a Service Provider.

What you need for this book

Windows, Mac OS X, or Linux

Node 0.6.x or Node 0.8.x freely obtained from http://www.nodejs.org

Will continue to be relevant on Node's 1.x.x release

Who this book is for

If you have some knowledge of JavaScript and want to build fast, efficient, scalable client-server solutions, then Node Cookbook is for you. Experienced users of Node will improve their skills, and even if you have not worked with Node before, these practical recipes will make it easy to get started.

Conventions

In this book, you will find a number of styles of text that distinguish between different kinds of information. Here are some examples of these styles, and an explanation of their meaning.

Code words in text are shown as follows: In order to create the server we need the http module.

A block of code is set as follows:

  var http = require('http');

  http.createServer(function (request, response) {

  response.writeHead(200, {'Content-Type': 'text/html'});

  response.end('Woohoo!');

  }).listen(8080);

When we wish to draw your attention to a particular part of a code block, the relevant lines or items are set in bold:

  var http = require('http');

 

var path = require('path');

 

  http.createServer(function (request, response) {

 

var lookup=path.basename(decodeURI(request.url));

Any command-line input or output is written as follows:

sudo npm -g install express

New terms and important words are shown in bold. Words that you see on the screen, in menus or dialog boxes for example, appear in the text like this: We could make it possible for a hypothetical user to indicate if they were inspired by a quote, such as a Like button.

Note

Warnings or important notes appear in a box like this.

Tip

Tips and tricks appear like this.

Reader feedback

Feedback from our readers is always welcome. Let us know what you think about this book — what you liked or may have disliked. Reader feedback is important for us to develop titles that you really get the most out of.

To send us general feedback, simply send an e-mail to <[e-mail]>, and mention the book title via the subject of your message.

If there is a book that you need and would like to see us publish, please send us a note in the SUGGEST A TITLE form on www.packtpub.com or e-mail .

If there is a topic that you have expertise in and you are interested in either writing or contributing to a book, see our author guide on www.packtpub.com/authors.

Customer support

Now that you are the proud owner of a Packt book, we have a number of things to help you to get the most from your purchase.

Downloading the example code

You can download the example code files for all Packt books you have purchased from your account at http://www.PacktPub.com. If you purchased this book elsewhere, you can visit http://www.PacktPub.com/support and register to have the files e-mailed directly to you.

Errata

Although we have taken every care to ensure the accuracy of our content, mistakes do happen. If you find a mistake in one of our books — maybe a mistake in the text or the code — we would be grateful if you would report this to us. By doing so, you can save other readers from frustration and help us improve subsequent versions of this book. If you find any errata, please report them by visiting http://www.packtpub.com/support, selecting your book, clicking on the errata submission form link, and entering the details of your errata. Once your errata are verified, your submission will be accepted and the errata will be uploaded on our website, or added to any list of existing errata, under the Errata section of that title. Any existing errata can be viewed by selecting your title from http://www.packtpub.com/support.

Piracy

Piracy of copyright material on the Internet is an ongoing problem across all media. At Packt, we take the protection of our copyright and licenses very seriously. If you come across any illegal copies of our works, in any form, on the Internet, please provide us with the location address or website name immediately so that we can pursue a remedy.

Please contact us at <[e-mail]> with a link to the suspected pirated material.

We appreciate your help in protecting our authors, and our ability to bring you valuable content.

Questions

You can contact us at <[e-mail]> if you are having a problem with any aspect of the book, and we will do our best to address it.

Chapter 1. Making a Web Server

In this chapter we will cover:

Setting up a router

Serving static files

Caching content in memory for immediate delivery

Optimizing performance with streaming

Securing against filesystem hacking exploits

Introduction

One of the great qualities of Node is its simplicity. Unlike PHP or ASP there is no separation between the web server and code, nor do we have to customize large configuration files to get the behavior we want. With Node we can create the server, customize it, and deliver content all at the code level. This chapter demonstrates how to create a web server with Node and feed content through it, all while implementing security and performance enhancements to cater for various situations.

Setting up a router

In order to deliver web content we need to make a URI available. This recipe walks us through the creation of an HTTP server that exposes routes to the user.

Getting ready

First, let's create our server file. If our main purpose is to expose server functionality, it's general practice to call the file server.js, which we could put in a new folder. It's also a good idea to install and use hotnode:

sudo npm -g install hotnode

hotnode server.js

Hotnode will conveniently auto-restart the server when we save changes.

How to do it...

In order to create the server we need the http module, so let's load it and use the http.createServer method:

  var http = require('http');

  http.createServer(function (request, response) {

  response.writeHead(200, {'Content-Type': 'text/html'});

  response.end('Woohoo!');

  }).listen(8080);

Tip

Downloading the example code

You can download the example code files for all Packt books you have purchased from your account at http://www.PacktPub.com. If you purchased this book elsewhere, you can visit http://www.PacktPub.com/support and register to have the files emailed directly to you.

Now, if we save our file and access localhost:8080 on a web browser or using curl, our browser (or curl) will exclaim:'Woohoo!'. However, the same will occur at localhost:8080/foo. Indeed, any path will render the same behavior, so let's build in some routing. We can use the path module to extract basename of the path (the final part of the path), and reverse any URI encoding from the client with decodeURI:

  var http = require('http');

 

var path = require('path');

 

  http.createServer(function (request, response) {

 

var lookup = path.basename(decodeURI(request.url));

We now need a way to define our routes. One option is to use an array of objects:

  var pages = [

   {route: '', output: 'Woohoo!'},

   {route: 'about', output: 'A simple routing with Node example'},

   {route: 'another page', output: function() {return 'Here\'s '+this.route;}},

  ];

Our pages array should be placed above the http.createServer call.

Within our server, we need to loop through our array and see if the lookup variable matches any of our routes. If it does we can supply the output. We'll also implement some 404 handling:

  http.createServer(function (request, response) {

   var lookup=path.basename(decodeURI(request.url));

  

pages.forEach(function(page) {

     if (page.route === lookup) {

       response.writeHead(200, {'Content-Type': 'text/html'});

       response.end(typeof page.output === 'function'

                     ? page.output() : page.output);

     }

   });

   if (!response.finished) {

       response.writeHead(404);

       response.end('Page Not Found!');

   }

 

  }).listen(8080);

How it works...

The callback function we provide to http.createServer gives us all the functionality we need to interact with our server through the request and response objects. We use request to obtain the requested URL and then we acquire its basename with path. We also use decodeURI which our another page route would fail without as our code would try to match another%20page against our pages array and return false.

Once we have our basename, we can match it in any way we want. We could send it in a database query to retrieve content, use regular expressions to effectuate partial matches, or we could match it to a file name and load its contents.

We could have used a switch statement to handle routing but our pages array has several advantages. It's easier to read and extend, and it can be seamlessly converted to JSON. We loop through our pages array using forEach.

Node is built on Google's V8 engine, which provides us with a number of ECMAScript 5 features. These features can't be used in all browsers as they're not yet universally implemented, but using them in Node is no problem! forEach is an ES5 implementation, but the ES3 way is to use the less convenient for loop.

While looping through each object, we check its route property. If we get a match, we write the 200 OK status and content-type headers. We then end the response with the object's output property.

response.end allows us to pass a parameter to it, which it writes just before finishing