# Test Driven Development With Node, Postgres, and Knex (Red/Green/Refactor)

Today we will be developing a RESTful API with Node, Express, Knex - a SQL query builder - and PostgreSQL using test driven development (TDD).

This post assumes prior knowledge of:

• SQL
• Node/Express
• NPM Packages

## Getting Started

Before we can start testing and writing code we need to set up our project, a database, and all the required dependencies…

### Project Setup

First, we need to create a basic boilerplate Express application. To do this, first install the Express-Generator globally:

We can now generate a basic Express application boilerplate:

Run npm start to ensure the application works. Once the server is running, navigate to http://localhost:3000/, and you should see ‘Welcome to Express’ on the main page.

### Database Setup

Start by installing PostgreSQL from the official download page.

If you’re on a Mac we recommend using Postgress.app.

As noted, we’ll be using Knex to interact with our database. Knex is a SQL query builder that we can use with PostgreSQL to handle migrations, manage the schema, and query the database.

Let’s start by installing Knex and pg, a module for interacting with Postgres.

Next, we need to create two new databases, one for developing and the other for testing. Open psql in the terminal, and create a new database:

With our database created we can initialize Knex.

### Knex Setup

Run the following command to create knexfile.js, the Knex configuration file:

Update the default info to:

This sets up three different settings for our databases:

1. test - for testing on the local environment
2. development - for developing, again on the local environment
3. production - for the production environment

Now, we can add schema migrations. Migrations allow us to define and update the database schema. We can create migrations in the terminal like so:

Now, knex has automatically added in a “db/migrations” folder, with a timestamped file inside of it. Here is where we define our schema. It should just contain two empty functions at the moment.

Let’s add in our code to create and drop tables.

Here, the up function creates the shows table while the down function drops the table. So we now have a schema defined, and a migration file ready to create that schema.

Create a new file called knex.js inside the “db” folder. In this file we specify the environment (test, development, or production), require the knexfile.js, and export the configuration (based on the environment) for our database:

Apply the migrations to both databases:

### Knex Seeds

Seeding is simply the process of populating the database with initial data. Knex utilizes seed files for this.

Run the following in your terminal to create a seed for development:

This will generate a folder called “seeds/development” in the “db” directory of your project, and in that file there will be a boilerplate setup for inserting data into the database:

Let’s change the file so we’re inserting relevant data. Notice how there’s also built-in promises so that the data will be seeded in the order that we specify:

Since JavaScript is asynchronous, the order that data is inserted can sometimes change. We want to make sure that the data is in the same order each time we run our seed file(s).

Run the seed file:

Before moving on, follow the same process for the test seed. Just use the same data as the development seed.

### Mocha/Chai Setup

With the database set up with data in it, we can start setting up our tests. Start by installing Mocha (test runner) and Chai (assertion) as well as ChaiHTTP (HTTP request module for integration testing). Make sure to also install mocha globally, so that we can run tests from the command line.

By default, Mocha searches for tests with a “test” folder.

This configuration can be changed with a mocha.opts file

Add a “test” folder to the root directory, and in that folder add a file called routes.spec.js. Then update routes/index.js:

Then within app.js update this line-

-to-

Now every single route in that file will be prefixed with ‘/api/v1’ Try it out. Fire up the server, and navigate to http://localhost:3000/api/v1/shows. You should see the string ‘send shows back’ in the browser.

Finally, update this line in app.js-

-to-

This prevents application log messages from displaying in the stdout when the tests are ran, making it much easier to read the output.

And make sure the error handlers return JSON:

## Developing via TDD

The premise behind Test Driven Development (TDD) is that you write tests first that fail which you then make pass. This process is often referred to as Red/Green/Refactor.

### Test Setup

In our test file, we’ll need to start by including the necessary requirements for testing:

The first line sets the NODE_ENV to test so that the correct Knex config is used from knexfile.js. The next line requires chai, the assertion module, giving us access to all the chai methods - like should().

By utilizing should() we are using the should assertion style. This is a personal preference. You could also use expect or assert.

Then we require chai-http. This module allows us make http requests from within our test file. Next, we link to our app so that we can test the request-response cycle. Finally, the describe block underneath the requirements is the wrapper for the tests. Keep in mind that you can nest describe blocks to better organize your test structure by grouping similar tests together.

### GET all shows

#### Red

In the first test case, which is nested inside the first describe block, we want to get ALL shows in our database:

So, we have a describe block, and within that block, we have a single it statement. An it statement defines a specific test case. Here we hit the route ‘/api/v1/shows’ with a GET request and test that the actual response is the same as the expected response.

Let’s break this test down…

First, by removing the test conditions, we can look at the basic test structure:

Since this is an asynchronous test, we need some way of telling the callback function that the test is complete. This is where the done() callback method comes into play. Once called (or if a two second timer is exceeded), Mocha knows that the test is finished running, and it can move on to the next test.

Now let’s look at the assertions:

The first thing we generally want to do, is test that the response has a status of 200. After that, these tests will change depending on what we return in the route handler. In this case, we are expecting that the content type is JSON and that the response body will be an array (of objects) and have a length equal to four (since there are four rows in the database). Finally, we are testing the keys and values within the first object of the array.

Try it out:

If all went well you should see this:

Essentially, the second assertion - res.should.be.json; - failed since we are sending plain text back. This is good! Remember: Red-Green-Refactor!

We just need to update the route to get the test to pass.

#### Green

Before updating the route, let’s create a queries module for handling, well, the database queries. Create a new file called queries.js with the “db” folder, and add the following code:

Here, we made a reference to our database via the Knex config file, added a helper function for simplifying each individual query, and finally queried the database to get ALL shows.

Update the route:

Run mocha again and see what happens:

Awesome! Just don’t forget the last step - refactor.

#### Refactor

What’s happening in the test database?

Since we seeded the database earlier, there’s data already in there, which could affect other tests (especially when rows are added, changed, and/or dropped). In the test, we are asserting that the length of the array is four. Well, if we add an item then that’s going to change the length, and that first test will fail.

Tests should be isolated from each other. So, we really should rollback the migrations before and after each test is ran, and then apply the migrations and re-seed the database before the next test is ran.

This is where beforeEach and afterEach come into play:

Now, the migrations will run and the database will be re-seeded before each nested describe block, and the migrations will be rolled back after each block (which will also drop the data).

Why rollback before each test? If any errors occur during a test, it won’t reach the afterEach block. So we want to make sure that if an error occurs we still rollback the database.

Run the tests again:

1 down, 4 to go!!!

Did you notice how the overall time is slightly slower? 325ms vs 164ms. This is because of the beforeEach and afterEach. Think about what’s happening, and why this would slow down the tests.

### GET single show

We have our route and test built to get All shows, so the next step is to just get one show back.

#### Red

Based on our test seed, the first show that should (you never know for certain with async code) is Suits:

We can write out a test for a new route that will return just a single show and the meta information about it. Remember our last test? It returned an array of objects. This time it should be a single object since we will be searching for a single item in the database.

This is very similar to the previous test. We’re still testing for a status code of 200, and the response should be JSON. This time, we expect that res.body is an object. Each of the properties afterwards should be the properties of the item with id ‘1’ in the database. So now if we run the tests, the first assertion should fail because we haven’t written our route yet:

#### Green

Add the query to queries.js, making sure to update module.exports:

Then build out the route:

Now run mocha, and let’s see if that worked:

Two routes down, two tests passing.

### POST

We now want to add an item to our database.

#### Red

For time’s sake, write the test assuming you will get a JSON object back that contains the data added to the database:

You can see here that our test block is slightly different than the previous two since we need to send information with the request to replicate how a client might send information to the server.

#### Green

With the test written and failing (did you remember to run the tests?), we can write the query and add the route (notice the pattern yet?).

Query:

Route:

.insert() returns an array containing the unique ID of the newly added item, so in order to return the actual data, we utilized the getSingle() query. This also ensures that the data has been inserted into the database correctly.

Do the tests pass?

Excellent. Just two routes left to go.

### PUT

We need to test the edit route.

#### Red

Similar to our POST route we will need to send data to the server. In this case, we’ll utilize the ID of an existing show in the database and send along an object with the updated fields. Then we’ll assert that the show has been updated correctly.

So here we are stating that the response body should contain the updated object from the database.

#### Green

You know the drill - Start with the query:

Then update the route:

Here, we again make two calls to the database. Once we’ve updated the item, we then nest another query to get that same item - which we then check to ensure that it has in fact been updated correctly.

The tests should pass.

#### Refactor

What happens if we try to change the ID? Update the test:

Run the tests now and they should fail:

Why? Because the updated ID of the test does not equal the ID passed in as a query parameter. What does this all mean? The unique ID should never change (unless it’s removed altogether).

Now, let’s revert the changes in the test, by removing id: 20,, and add a new test:

Run the tests:

Boom!

### DELETE

Now on to the final test - the delete.

#### Red

Again, let’s use the ID of the first item in our database as the starting point for the test:

The test ensure that the deleted show is returned and that the database no longer contains the show.

#### Green

Query:

Route:

The Knex delete() function returns a number indicating the number of rows in the database that have been affected, so to return the deleted object, we must query for it first.

Let’s run those tests!!

6 tests written. 5 routes built. All tests passing!

## Conclusion

So there you have it: A test-first approach to developing a RESTful API. Are we done? Not quite since we are not handling or testing for all possible errors.

For example, what would happen if we tried to POST an item without all the required fields? Or if we tried to delete an item that isn’t in the database? Sure the catch() methods will handle these, but they are simply passing the request to the built-in error handlers. We should handle these better in the routes and throw back appropriate error messages and status codes.

Try this out on your own. Be sure to grab the code from the repository. Cheers!

Edits made by Bradley Bouley. Thank you!