nodejs-mongodb_thv5ac

in Jixee Hotfix

How to Write a REST API with MongoDB, NodeJS, Express, and Simple JSON Web Token Part 1

[Editor’s Note: We’re starting a new series on this blog, called Jixee Hotfix. It will feature real problems that our engineering team encounter on a weekly basis and the solutions they come up with to fix it. Posts are written by the engineers encountering the problems. This post was written by our VP of Ops, Eric Norton.]

This article is the first in a series that will show you how to write a REST API written in NodeJS and Express, that uses MongoDB to store data, and JSON Web Tokens(JWT)  to provide a simple authentication mechanism.  For those who came here wanting to learn about JSON Web Tokens (JWT) authentication, that is covered in part 2 here.  This installment of the series will only cover details on how to create a REST API using MongoDB as a persistent data store.

You might be asking yourself, ‘why another REST API tutorial?’ With a substantial amount of great articles out there on the subject, it’s a fair question.  Let me give a little background on this project to explain.  A task fell in my lap a short time ago that required a simple REST API to import and export data stored by one of our services.   In my search for a simple solution, I found many great blog articles that covered some of the concepts I was interested in, but not all.  For instance, some discussed building APIs that stored and retrieved data, and some discussed simple JWT auth, but did not cover how you would incorporate a persistent data store.   I decided that I’d like to write an article that combines all of the concepts I was looking for into one concise resource.  While I will cover a lot of the concepts I learned in the aforementioned articles, I encourage you to take a look at what inspired this post, as they are great resources on the subject matter:

Creating A Simple Restful Web App with NodeJS, Express, and MongoDB
REST follow-up exercise, implementing a PUT into a simple web app
Architecting a Secure RESTful Node.js app
Express.js 4, Node.js and MongoDB REST API Tutorial
Build a RESTful API in 5 Minutes with NodeJS – Updated

Alright, grab your caffeine and let’s get started. This tutorial assumes you have a novice understanding of NodeJS and have the following requirements installed on your machine:

  • mongodb
  • nodejs
  • npm
  • nodejs modules:
    • express
    • body-parser
    • mongoose
    • morgan
    • nodemon

If you’re an advanced user, you’ll find that you can follow this tutorial on any platform, but my development platform is Ubuntu 14.04, so I’ll be providing examples of how to install the requirements on that platform. If you’d like to follow along in the same environment, fire up your favorite hypervisor, and install Ubuntu 14.04 server and run the following commands to fulfill the requirements:

Let’s install MongoDB first:

sudo apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv 7F0CEB10
echo 'deb http://downloads-distro.mongodb.org/repo/ubuntu-upstart dist 10gen' | sudo tee /etc/apt/sources.list.d/mongodb.list
sudo apt-get update
sudo apt-get install -y mongodb-org

Now start MongoDB:

sudo start mongod

Now let’s install NodeJS:

sudo apt-get install nodejs
sudo apt-get install npm

We’ll be using cURL to talk to our API so let’s make sure that’s installed:

sudo apt-get install curl

Now let’s make the directories we’re going to use to store our project and files:

mkdir -p ~/restapi/config
mkdir -p ~/restapi/dao
mkdir -p ~/restapi/models
mkdir -p ~/restapi/routes

We’ll discuss what goes into these directories in a moment.

The first thing we’ll want to do is create a nodejs package using npm.  This process will ask you a few questions about your project and create a file called package.json in the root of your package directory. This file contains the details of the package you’re creating, as well as dependencies for the package.

Run:

cd ~/restapi
npm init

After running the command, here’s what it’ll look like.  I’ve answered the questions in red and kept the defaults (simply hit enter) for the questions in blue:

This utility will walk you through creating a package.json file.
It only covers the most common items, and tries to guess sane defaults.
See `npm help json` for definitive documentation on these fields and exactly what they do.
Use `npm install <pkg> --save` afterwards to install a package and save it as a dependency in the package.json file.
Press ^C at any time to quit.
name: (restapi)
version: (0.0.0) 0.0.1
description: Simple REST API with persistent data store
entry point: (index.js) server.js
test command:
git repository:
keywords: rest api
author: Eric Norton
license: (BSD-2-Clause) MIT
About to write to /home/eric/restapi/package.json:
{
 "name": "restapi",
 "version": "0.0.1",
 "description": "Simple REST API with persistent data store",
 "main": "server.js",
 "scripts": {
 "test": "echo \"Error: no test specified\" && exit 1"
 },
 "keywords": [
 "rest",
 "api"
 ],
 "author": "Eric Norton",
 "license": "MIT"
 }

 

Let’s install the nodejs modules we want with a flag that adds the dependency to our package.json file.

npm install body-parser --save
npm install express --save
npm install mongoose --save
npm install morgan --save

We’ll install the following module globally because we’ll want it available to our other projects:

sudo npm install nodemon -g

The beauty of having a package.json file means that you can run the following command from within your package directory at any time, and it will install all of your nodejs module depencies:

npm install

If you are using GIT as your version control, it makes sense to exclude the node_modules directory from being committed to your repo by adding the following to your .gitignore file in the root:

node_modules

There are many ways to deal with node_modules in your repository, but I’ve found keeping the modules from being committed to the package repository keeps things organized, minimizes the noise in your commit logs, and makes it easier to fix merge conflicts when they happen.

Alright, we’re ready to add some code. It’s worth noting that you can find all of the code for this project in my github account here(https://github.com/nortone/restapi.git).  In this tutorial, we’re going to build a REST API that allows us to Create, Read, Update, and Delete (CRUD) beers we’re aging in our cellar.  I had you create a directory structure above where we’ll be storing our files.  Let’s go through the directories and explain how we’ll be organizing our code.

 

Here’s our structure:

~restapi/
      |
    config/     →  where we’ll store our configs
    dao/          →  where we’ll store our data access objects
    models/   →  where we’ll store our data models
    routes/     →  our API routes will go here

 

Now that we know how we’ll be organizing our files, the first thing we’ll want to do is create our server.js file.  This is the file we declared in our package.json as the ‘main’ script that starts our app. Let’s create the following file:

~/restapi/server.js

With this code:

var express = require('express');
var path = require('path');
var logger = require('morgan');
var bodyParser = require('body-parser');
var config = require('./config/config');
 
var app = express();
 
var mongoose = require('mongoose');
var db = mongoose.connection;
 
// connect to the db
mongoose.connect(config.db);
 
app.use(logger('dev'));
app.use(bodyParser.json());
 
app.all('/*', function(req, res, next) {
  // CORS headers
  res.header("Access-Control-Allow-Origin", "*"); // restrict it to the required domain
  res.header('Access-Control-Allow-Methods', 'GET,PUT,POST,DELETE,OPTIONS');
  // Set custom headers for CORS
  res.header('Access-Control-Allow-Headers', 'Content-type,Accept');
  // If someone calls with method OPTIONS, let's display the allowed methods on our API
  if (req.method == 'OPTIONS') {
    res.status(200);
    res.write("Allow: GET,PUT,POST,DELETE,OPTIONS");
    res.end();
  } else {
    next();
  }
});
 
// start db
db.on('error', console.error);
db.once('open', function() {
 
  app.use('/', require('./routes'));
 
  // If no route is matched, return a 404
  app.use(function(req, res, next) {
    var err = new Error('Not Found');
    err.status = 404;
    next(err);
  });
 
  // Start the server
  app.set('port', process.env.PORT || 3000);
 
  var server = app.listen(app.get('port'), function() {
    console.log('Express server listening on port ' + server.address().port);
  });
});

Let’s briefly go through parts of the code to explain what’s going on.  This is often not covered in REST API tutorials, but a method that gets overlooked is the method OPTIONS.  This method allows the user to make an OPTIONS call to an API and get a response of what methods are allowed for a certain resource.  For illustration purposes, we’ll keep it simple and always return the allowed methods for the whole API anytime the OPTIONS method is called.   Here’s the snippet(lines 25-31):

  if (req.method == 'OPTIONS') {
    res.status(200);
    res.write("Allow: GET,PUT,POST,DELETE,OPTIONS");
    res.end();
  } else {
    next();
  }

Since this file will be the file we execute to start our app, we’ll include the global headers for all of our routes.  You can see that in this block of code(lines 18-32):

app.all('/*', function(req, res, next) {
  // CORS headers
  res.header("Access-Control-Allow-Origin", "*"); // restrict it to the required domain
  res.header('Access-Control-Allow-Methods', 'GET,PUT,POST,DELETE,OPTIONS');
  // Set custom headers for CORS
  res.header('Access-Control-Allow-Headers', 'Content-type,Accept');
  // If someone calls with method OPTIONS, let's display the allowed methods on our API
  if (req.method == 'OPTIONS') {
    res.status(200);
    res.write("Allow: GET,PUT,POST,DELETE,OPTIONS");
    res.end();
  } else {
    next();
  }
});

Another block of code to note is where we start our database. In this block is where we’ll define our routes and what to do with them if they don’t exist. Notice that we are including the bulk of our routes with a require line(lines 36-53):

db.once('open', function() {
 
  app.use('/', require('./routes'));
 
  // If no route is matched, return a 404
  app.use(function(req, res, next) {
    var err = new Error('Not Found');
    err.status = 404;
    next(err);
  });
 
  // Start the server
  app.set('port', process.env.PORT || 3000);
 
  var server = app.listen(app.get('port'), function() {
    console.log('Express server listening on port ' + server.address().port);
  });
});

We have a few more files to create before we can start our application. Next we’ll need to create a data model for our data using mongoose.  We’ll create that file here:

~/restapi/models/beer.js

With the code:

var mongoose = require('mongoose');
 
var beerSchema = mongoose.Schema({
  beerid: String,
  beer: String,
  brewery: String,
  abv: String,
  year: String,
  cellardate: Date,
  style: String,
  description: String,
  notes: String,
  total: Number
});
 
module.exports = mongoose.model('Beer', beerSchema);

This code utilizes the node module Mongoose, to help us easily create a data model for our Beer object. Note that I’ve use 3 different data types. These types are important when dictating the type of data we’ll be storing.

Now that we have our data model, let’s create a data access object for our Beers.  This file will go here:

~/restapi/dao/beers.js

With the following code:

var Beer = require('../models/beer');
var beers = {
 
  getAll: function(req, res) {
    Beer.find(function (err,beers) {
      if (err) {
        console.log(err);
      } else {
        res.send(beers);
      }
    });
  },
 
  getOne: function(req, res) {
    var id = req.params.id;
    Beer.findOne({ beerid: id }, function (err,beer) {
      if (err) {
        console.log(err);
      } else {
        if (beer) {
          res.send(beer);
        } else {
          res.status(404);
          res.json({
            "status": 404,
            "message": "Not Found"
          });
        }
      }
    });
  },
 
  create: function(req, res) {
    var body = req.body;
    Beer.findOne({ beerid: body.beerid }, function (err,beer) {
      if (err) {
        console.log(err);
      } else {
        if (beer) {
          res.status(409);
          res.json({
            "status": 409,
            "message": "Beer already exists."
          });
        } else {
          var newBeer = new Beer({
            beerid: body.beerid,
            beername: body.beername,
            brewery: body.brewery,
            abv: body.abv,
            year: body.year,
            cellardate: body.cellardate,
            style: body.style,
            description: body.description,
            notes: body.notes,
            total: body.total
          });
          newBeer.save(function(err,newBeer) {
            if (err) {
              return console.error(err);
            } else {
              res.json(newBeer);
            }
          });
        }
      }
    });
  },
 
  update: function(req, res) {
    var body = req.body;
    var id = req.params.id;
 
    Beer.findOne({ beerid: id }, function (err,beer) {
      if (err) {
        console.log(err);
      } else {
        if (beer) {
          Beer.findOneAndUpdate({beerid:id},body, function (err,updatedbeer) {
            if (err) {
              console.log(err);
            } else {
              res.json(updatedbeer);
            }
          });
        } else {
          res.status(404);
          res.json({
            "status": 404,
            "message": "Not Found"
          });
        }
      }
    });
 
  },
 
  delete: function(req, res) {
    var id = req.params.id;
    Beer.findOne({ beerid: id }, function (err,beer) {
      if (err) {
        console.log(err);
      } else {
        if (beer) {
          Beer.remove({beerid: id}, function (err,beer) {
            if (err) {
              console.log(err);
            } else {
              // normally we would return a 'true' or 'false' to our client, but let's output a status
              // for illustration purposes
              res.status(200);
              res.json({
                "status": 200,
                "message": "delete of " + id + " succeeded."
              });
            }
          });
        } else {
          res.status(404);
          res.json({
            "status": 404,
            "message": "Not Found"
          });
        }
      }
    });
  }
};
 
module.exports = beers;

At quick glance, this file looks complicated, but if we break it up into parts, you’ll see that our data access object contains 5 functions:

 

  • getall()    → will read all of our beers from the database and return them
  • getOne() → will read one beer from the database and return it
  • create()   → will add a beer to the database
  • update()  → will update a beer already in the database
  • delete()   → will delete a beer from the database

 

In each function is a small block of code that will use the data model we created to perform CRUD functions on our data.  All of these functions use the Mongoose interface to work with the data.  I’ve saved you the trouble of looking up which Mongoose methods are used to read and write data to and from our MongoDB database, but you can find more detailed information about the specific Mongoose API calls I’m using in the Mongoose documentation here.

Alright, we’re getting closer.  Now we need to create routes that interact with our data access object beers.js.

Let’s create this file:

1
~/restapi/routes/index.js

And add this code:

var express = require('express');
var router = express.Router();
 
var beer = require('../dao/beers.js');
 
/*
 * Routes
 */
router.get('/beers', beer.getAll);
router.get('/beer/:id', beer.getOne);
router.post('/beer/', beer.create);
router.put('/beer/:id', beer.update);
router.delete('/beer/:id', beer.delete);
 
module.exports = router;

As you can see, this file is short and simple, but don’t be fooled, this file is very important in letting our application know what data access functions correspond to what route. This is where a lot of the magic happens, so we’ll go over each line one by one.

 

router.get('/beers', beer.getAll);

This sets a route with method GET from the root path called /beers, and anytime this route is called, it will use the function getAll() in our data access object and return all the beers in our database.

 

router.get('/beer/:id', beer.getOne);

This sets a route with method GET from the root called /beers/:id. The :id is a dynamic parameter that corresponds to ‘beerid’ in our data model.  This will use the getOne() function in  our data access object and return the object of the specified :id if it exists.

 

router.post('/beer/', beer.create);

This sets a route with method POST that allows us to send a JSON object containing all of the items in our beer data model. This route will call create() from our data access object and add the JSON object to our database.

 

router.put('/beer/:id', beer.update);

This sets a route with method PUT that allows us to send a JSON object containing updated data values for our given :id. This route calls update() from our data access object and will return the JSON of the updated beer if successful.

 

router.delete('/beer/:id', beer.delete);

This sets a route with method DELETE that allows us to delete a specified :id. Usually I like returning true or false here to make it easier for clients to see whether a delete was successful or not, but in this code example, we’ll just return status 200 and a success message for illustration purposes.

 

It’s valuable to note that if you called any of these routes with a method other than the one you’ve set per route, you’ll get a 404. For instance, you’ll never be able to use PUT to create an object, or you’ll never be able to use POST to get a list of all your objects. The reason for this is simple, we haven’t defined those routes in this file.

Alright, the hardest part is done.  Now, let’s finish up our code by adding a config.  Because this app is so simple, we could technically omit this part, but it’s good to get into the habit of using configs in your apps to make them easier to deploy to different environments(ie. staging, production).

Create this file:

~/restapi/config/config.js

And add this code to it:

// define config object
var config = {};
 
// mongodb connection
config.db = "mongodb://localhost/restapi";
 
module.exports = config;

This is the simplest file in our app. It creates a config object, defines the database URL we’ll be using, and then exports the config object.

Pat yourself on the back, because at this point, you’ve successfully written a simple REST API.  Now for the fun part, let’s see how it works.

Earlier in this article, we installed the node module nodemon globally.  If you aren’t already familiar with nodemon, it’s a node module that makes developing nodejs apps easier.  It will restart your app for you when it detects any changes in your code, and makes sure your app stays up and running.  Let’s fire it up:

cd ~/restapi
nodemon

You’ll see something like this as output:

26 Jan 18:47:14 - [nodemon] v1.2.1
26 Jan 18:47:14 - [nodemon] to restart at any time, enter `rs`
26 Jan 18:47:14 - [nodemon] watching: *.*
26 Jan 18:47:14 - [nodemon] starting `node server.js`
Express server listening on port 3000

If you see the above output then your app started!  On certain platforms with older versions of Python you might see the following Error message:

{ [Error: Cannot find module '../build/Release/bson'] code: 'MODULE_NOT_FOUND' }
js-bson: Failed to load c++ bson extension, using pure JS version

The message is telling us that we don’t have the native mongodb driver installed. It’s ok to ignore. For this tutorial, the JS version will suffice.

Alright let’s use our API.  The first thing we’ll want to do is add some beers to our database.  We can do that using the curl command:

curl -X POST -H "Content-Type: application/json" http://localhost:3000/beer -d '{
"beerid":"bcstout",
"beername":"Bourbon Country Brand Stout",
"brewery":"Goose Island",
"abv":"13.80",
"year":"2014",
"cellardate":"Mon Sep 08 2014",
"style":"Imperial Stout",
"description":"bourbon barrel aged",
"notes":"store at 55 degrees celsius",
"total":4
}'

This is what you should see after running the above command:

{"__v":0,"beerid":"bcstout","brewery":"Goose Island","abv":"13.80","year":"2014","cellardate":"2014-09-08T07:00:00.000Z","style":"Imperial Stout","description":"bourbon barrel aged","notes":"store at 55 degrees celsius","total":4,"_id":"54c71395e63d39743df82f2e"}

Now let’s run that command again exactly as it is above.  If the command worked the first time around, you should see:

{"status":409,"message":"Beer already exists."}

We’ve added some logic in our beer data access object for the create() function that makes sure it doesn’t create a new object if the same beerid already exists. This is just one example of logic you can add to your data access functions that decide what to do with the data given to a function or route.

Now let’s add a couple more beers to our database so we have a few objects to work with:

curl -X POST -H "Content-Type: application/json" http://localhost:3000/beer -d '{
"beerid":"blacktuesday",
"beername":"Black Tuesday",
"brewery":"The Bruery",
"abv":"19.20",
"year":"2013",
"cellardate":"Thu Aug 01 2013",
"style":"Imperial Stout",
"description":"bourbon barrel aged",
"notes":"store at 55 degrees celsius",
"total":2
}’
 
curl -X POST -H "Content-Type: application/json" http://localhost:3000/beer -d '{
"beerid":"parabola",
"beername":"Parabola",
"brewery":"Firestone Walker Brewing Co.",
"abv":"13.00",
"year":"2014",
"cellardate":"Tue Apr 15 2014",
"style":"Russian Imperial Stout",
"description":"bourbon barrel aged",
"notes":"store at 55 degrees celsius",
"total":2
}’

Now that we have a few beers in our database, let’s list them all:

curl -X GET http://localhost:3000/beers

You should see all of the objects jumbled together in a line like this:

[{"_id":"54c6ce5949038d6a3a050236","beerid":"blacktuesday","brewery":"The Bruery","abv":"19.20","year":"2013","cellardate":"2013-08-01T07:00:00.000Z","style":"Imperial Stout","description":"bourbon barrel aged","notes":"store at 55 degress celsius","total":2,"__v":0},{"_id":"54c71395e63d39743df82f2e","beerid":"bcstout","brewery":"Goose Island","abv":"13.80","year":"2014","cellardate":"2014-09-08T07:00:00.000Z","style":"Imperial Stout","description":"bourbon barrel aged","notes":"store at 55 degrees celsius","total":4,"__v":0},{"_id":"54c7158ee63d39743df82f2f","beerid":"parabola","brewery":"Firestone Walker Brewing Co.","abv":"13.00","year":"2014","cellardate":"2014-04-15T07:00:00.000Z","style":"Russian Imperial Stout","description":"bourbon barrel aged","notes":"store at 55 degrees celsius","total":2,"__v":0}]

This is obviously a bit hard to read, so you can also load that URL in a browser with a JSON output extension installed like JSONView or use a tool like Postman  for Chrome to work with the API.  These tools will format the JSON for readability so it comes out more like:

[
{
"_id": "54c6ce5949038d6a3a050236",
"beerid": "blacktuesday",
"brewery": "The Bruery",
"abv": "19.20",
"year": "2013",
"cellardate": "2013-08-01T07:00:00.000Z",
"style": "Imperial Stout",
"description": "bourbon barrel aged",
"notes": "store at 55 degress celsius",
"total": 2,
"__v": 0
},
{
"_id": "54c71395e63d39743df82f2e",
"beerid": "bcstout",
"brewery": "Goose Island",
"abv": "13.80",
"year": "2014",
"cellardate": "2014-09-08T07:00:00.000Z",
"style": "Imperial Stout",
"description": "bourbon barrel aged",
"notes": "store at 55 degrees celsius",
"total": 4,
"__v": 0
},
{
"_id": "54c7158ee63d39743df82f2f",
"beerid": "parabola",
"brewery": "Firestone Walker Brewing Co.",
"abv": "13.00",
"year": "2014",
"cellardate": "2014-04-15T07:00:00.000Z",
"style": "Russian Imperial Stout",
"description": "bourbon barrel aged",
"notes": "store at 55 degrees celsius",
"total": 2,
"__v": 0
}
]

Alright, let’s say we want to add another Bourbon County Brand Stout to our cellar (represented by the ‘total’ variable). This command is very similar to the POST we just did when we created a beer, only we’re going to use the PUT method and specify a ‘beerid’ like so:

curl -X PUT -H "Content-Type: application/json" http://localhost:3000/beer/bcstout -d '{
"beerid":"bcstout",
"beername":"Bourbon Country Brand Stout",
"brewery":"Goose Island",
"abv":"13.80",
"year":"2014",
"cellardate":"Mon Sep 08 2014",
"style":"Imperial Stout",
"description":"bourbon barrel aged",
"notes":"store at 55 degrees celsius",
"total":6
}'

You should see the same output similar to this:

{"_id":"54c71395e63d39743df82f2e","beerid":"bcstout","brewery":"Goose Island","abv":"13.80","year":"2014","cellardate":"2014-09-08T07:00:00.000Z","style":"Imperial Stout","description":"bourbon barrel aged","notes":"store at 55 degrees celsius","total":6,"__v":0}

Easy enough right?

So up to this point we can add beers, display them, and update them, so let’s celebrate with our friends by drinking all of our Bourbon County Brand Stouts, and delete them from the database.

curl -X DELETE http://localhost:3000/beer/bcstout

If all goes well, you should see:

{"status":200,"message":"delete of bcstout succeeded."}

And there you have it, you’ve successfully written a simple REST API that stores data in MongoDB and let’s you access it using standard http methods.   I encourage you to play with the API and see what it can do.

As mentioned above, all of the code for this tutorial can be found in my git repository here: https://github.com/nortone/restapi.git

I’ve separated the articles into parts so the directory ‘part1’ covers everything we did in this post.  Stay tuned for part 2 where we’ll expand our beer database by adding users and simple JWT authentication.

Thanks for reading!  Comments are welcome.