“When you find you have to add a feature to a program, and the program’s code is not structured in a convenient way to add the feature, first refactor the program to make it easy to add the feature, then add the feature.”
Martin Fowler, Refactoring: Improving the Design of Existing Code
Introduction
Hi there! This is the fifth part of “How to Create REST API Using Node.js” post series. We sure have learnt a lot from the previous tutorials. Should you stumble to this post first, here’s the link to the previous posts, and it is recommended that you read them first:
I will share about refactoring the code we created on the previous posts. Why refactor? The code works alright. Because it has become more of a mess with each feature addition.
At the end of the post, I will share the source code in GitHub as usual.
Requirements
A desire to make the code look better. In other words, no specific libraries needed for this part.
Oh, if you’d like to start fresh and follow this tutorial, you can clone from this repository.
Part 5 – REST API Code Refactor Because It Became Complicated Already
POST /user
Did you notice that the password is saved as is in the database? This is a vulnerability because a database admin can access the database and obtain any user’s password. If he’s up to no good, he could exploit the data to hack to the account and do any cyber crime he could think of.
The way to solve this is to add an encryption on the password on inserting a new data. We will use bcrypt to encrypt the password data. On the terminal, run this command: npm i bcrypt to install the bcrypt library to the project.
index.js
After installing the library, we modify our POST /user from app.post('user'). See this snippet on index.js:
// index.js
...
const config = {
app: {
...,
bcrypt: {
saltRounds: parseInt(env.SALT_ROUNDS)
}
},
...
};
const bcrypt = require('bcrypt');
const express = require('express');
...
app.post('/user',
configMiddleware,
authMiddleware,
async (req, res) => {
const payload = { ...req.body };
const { saltRounds } = config.app.bcrypt;
payload.password = await bcrypt.hash(payload.password, saltRounds);
const result = await mongoDbConnector.insertOne(collection, payload);
res.send(result);
}
);First we add a new environment variable saltRounds from .env.
Then, we import the bcrypt library. You can put it anywhere on top of the file.
Lastly, we modify app.post('user'). In the callback function, we modify the password payload before inserting it to the database. We hash the password using await bcrypt.hash() with two arguments: the password from req.body and saltRounds from config.
.env and .env.example
Then, on .env, add this new environment variable:
# .env
...
SALT_ROUNDS=10
...Test the code
There isn’t much to modify, so you should be able to do this correctly and quickly. Run npm start and it should look like this in the database when you hit POST /user like before:
POST /login
After hashing the passwords on register, we need to modify POST /login to check the verify the password correctly, because if we check it in the existing way, it will always return as incorrect for obvious reasons.
index.js
And so, let’s modify app.post("/login") in index.js to verify the hash. See this snippet below:
// index.js
...
app.post('/login', async (req, res) => {
const { username, password } = req.body;
if (username && password) {
const result = await mongoDbConnector.findOne(
collection,
{ username }
);
let verifyPass = false;
if (result.password) {
verifyPass = await bcrypt.compare(password, result.password);
}
if (verifyPass) {
...
}
...
}
});
...Inside app.post('/login') and inside the if (username && password) if block, change the filter on mongoDbConnector.findOne() to { username } only.
If a result is found, verify the plain text password with the hash with await bcrypt.compare(password, result.password). The expected result is true or false.
Lastly, change the if conditional of if (result !== 'Data not found.') to if (verifyPass).
Test the code
The changes here are simple, you should be able to get it right fast. Whenever you are ready, run npm start, then hit POST /login with the same body parameters. It should be like this screenshot below:
Note that you cannot login using the accounts with the previously un-hashed password anymore. Because it’s unsafe.
Move Config from index.js to config.js
Our config object, which contains loaded environment variables, are getting bigger and bigger with each feature updates. Though it is fine to put it on index.js, we want it to be tidier. So, let’s move them to config.js.
config.js
Move almost everything about config and create a new file named config.js. See this snippet:
const env = require('dotenv').config().parsed;
const config = {
app: {
host: env.APP_HOST,
port: env.APP_PORT,
auth: {
key: env.AUTH_KEY,
expiry: env.AUTH_EXPIRY
},
bcrypt: {
saltRounds: parseInt(env.SALT_ROUNDS)
}
},
db: {
host: env.DB_HOST,
name: env.DB_NAME,
collections: {
user: env.DB_COLLECTION_USERS
}
},
redis: {
host: env.REDIS_HOST,
port: env.REDIS_PORT,
expiry: env.REDIS_EXPIRY_SECS
}
};
module.exports = config;The codes moved are env and config, they are exactly the same. Then, add module.exports = config; to enable config.js to be imported from another module.
index.js
Simply import config and name it as config. It is important to import the config at the very top of the file, because the environment variables must be loaded first. See this snippet below:
// index.js
const config = require('./config');
...Test the code
Run npm start and random test the created URLs. If it works correctly, then all is well.
Create an initializer module
Some things such as MongoDbConnector, RedisConnector, and config are initialized once when the REST API server starts. They are currently placed in the beginning of index.js, which is a correct place. But it will get much longer with every update, making index.js more complicated.
initialize.js
With that in mind, we will separate those parts and create a initializer module, to load everything needed when starting the server. See this snippet below:
// initialize.js
const config = require('./config');
const MongoDbConnector = require('./mongoDbConnector');
const RedisConnector = require('./redisConnector');
const initMongoDb = () => {
const mongoDbConnector = new MongoDbConnector({
name: config.db.name,
host: config.db.host
});
mongoDbConnector.connect();
return mongoDbConnector;
}
const initRedis = () => {
const redisConnector = new RedisConnector();
redisConnector.connect(config.redis.host, config.redis.port);
return redisConnector;
}
const initialize = () => ({
mongoDbConnector: initMongoDb(),
redisConnector: initRedis(),
config,
collection: config.db.collections.user
});
module.exports = initialize;You have seen this code in index.js. We just move them to a new file called initialize.js and separate them as initMongoDb() and initRedis().
Then, we create a function to run all of them, then return all the initialized values as an object. Lastly, we add module.exports, so it can be imported from index.js.
In the future, initializing all services should be done from this module. If you want to, you may take one step further by creating a new directory consisting of initializing modules. For example, you may separate initMongoDb and initRedis in this case to separate files, then the main initialize.js will import them.
index.js
In index.js, remove all config, mongoDbConnector and redisConnector parts when initializing, and replace with these snippet:
const initialize = require('./initialize');
...
app.use(bodyParser.json());
const context = initialize();
const {
mongoDbConnector,
redisConnector,
config,
collection
} = context;
const configMiddleware = async (...) => { ... };
...First, we import initialize we created before. Then, skip to the part before configMiddleware. Create a context object by running initialize(). Lastly, we destructure context to get the connectors and config we need before.
Test the code
Run npm start, then hit any endpoint we created before. If it works, then all is well.
Change the way on creating middlewares
For middleware, currently we have authMiddleware only and we load it directly from index.js. What if we have multiple middlewares? We need to separate them from index.js.
Create new “middlewares” directory
First we create a new middlewares directory. Then, we put authMiddleware.js to the middlewares directory.
Modify authMiddleware
Next, we modify authMiddleware, so it can have context when executed. See this snippet below:
// middlewares/authMiddleware.js
const { verifyToken } = require('../authUtils');
const createAuthMiddleware = (context) => {
return async (req, res, next) => {
const { auth } = req.headers;
const { config } = context;
try {
verifyToken(auth, config.app.auth.key);
next();
}
catch (err) {
res.status(401).send(err);
}
}
}
module.exports = createAuthMiddleware;If you at it closely, the authMiddleware function is wrapped with another function called createAuthMiddleware(). This function requires context as parameter and will return the original authMiddleware function before.
The currying is necessary because there will be no other way to get config when the middleware is assigned on any of the routes. In other words, we don’t want to create the middleware which requires config in index.js later on.
Create a Middlewares Initializer
Then, we create a new file named middlewares/initializeMiddlewares.js. Here is where we will create new middlewares along with the context data inside them:
// middlewares/initializeMiddlewares.js
const createAuthMiddleware = require('./authMiddleware');
const initializeMiddlewares = (context) => ({
authMiddleware: createAuthMiddleware(context)
});
module.exports = initializeMiddlewares;Import authMiddleware and then create a new function named initializeMiddlewares with context as parameter. Lastly, return an object which consists of the created middlewares. Because there are only one middleware right now, so only authMiddleware is in the object.
In the future, you can put all new middlewares here to be initialized.
initialize.js
Next, we modify initialize.js to also initialize middlewares:
// initialize.js
const config = require('./config');
const MongoDbConnector = require('./mongoDbConnector');
const RedisConnector = require('./redisConnector');
const initializeMiddlewares = require('./middlewares/initializeMiddlewares');
...
const initialize = () => {
const context = {
mongoDbConnector: initMongoDb(),
redisConnector: initRedis(),
config,
collection: config.db.collections.user
};
const middlewares = initializeMiddlewares(context);
Object.assign(context, { middlewares });
return context;
}
module.exports = initialize;First, we import initializeMiddlewares, and then modify initialize() to add initialized middlewares. initializeMiddlewares() requires context on execution. After initializing middlewares, assign it as a property of context. Lastly, return context as the result of initialize().
index.js
Back to index.js, we will now change where we get authMiddleware. Instead of directly importing it, we will get authMiddleware from context, just like the connectors and config. See this snippet below:
// index.js
...
// remove this line
const authMiddleware = require('./middlewares/authMiddleware');
...
const context = initialize();
const {
...,
middlewares
} = context;
const { authMiddleware } = middlewares;
...
// remove this line, and all lines which has 'configMiddleware'
const configMiddleware = async (req, res, next) => {
Object.assign(req.app, { config });
next();
};
...First, we change the way to load authMiddleware by removing the authMiddleware which was imported directly. Then on the destructure block of context, add middlewares. Next, destructure middlewares to get authMiddleware.
And then, this is something I forgot, because we have config already in context, we have no need of configMiddleware. So, remove it. Also, remove every configMiddleware attached on the created routes.
Test the Code
Run npm start on your Terminal and try to hit any routes except POST /login. If there are errors thrown from JWT, then your changes work properly.
Separate handler on index.js to each file
There are still a lot going on in index.js. Mostly, it’s because of the handler functions on every route being put in the same file. Because there are many routes, I will share only an example of refactoring these handler functions.
First we create a new directory named handlers. Then, create a new file named initializeHandlers.jsand another named createUserHandler.js. It should look like this:
createUserHandler.js
Let’s get to createUserHandler first. See this snippet below:
// createUserHandler.js
const bcrypt = require('bcrypt');
const createCreateUserHandler = (context) => {
const { mongoDbConnector, config, collection } = context;
return async (req, res) => {
const payload = { ...req.body };
const { saltRounds } = config.app.bcrypt;
payload.password = await bcrypt.hash(payload.password, saltRounds);
const result = await mongoDbConnector.insertOne(collection, payload);
res.send(result);
}
}
module.exports = createCreateUserHandler;First, I would like you to see the return part. Are you reminded of this code somewhere? It’s the handler function of app.post("/user").
Just like authMiddleware before, we create a function to provide context, which has every data, to the handler function. That is why the function has the prefix create. And so, createCreateUserHandler returns a createUserHandler function to be used later on.
Oh, it also loads bcrypt library to hash the new passwords.
initalizeHandlers.js
Next, we write initializeHandlers.js to initialize all handlers specified here. See this snippet:
// initializeHandlers.js
const createUserHandler = require('./createUserHandler');
const updateUserHandler = require('./updateUserHandler');
...
const initializeHandlers = (context) => ({
createUserHandler: createUserHandler(context),
updateUserHandler: updateUserHandler(context),
...
});
module.exports = initializeHandlers;This function has the same structure as initializeMiddlewares.js. It accepts a context parameter, creates functions from loaded handlers, then returns them all as an object.
More handlers will be added here as you update the script.
initialize.js
Next, we import initializeHandlers.js to initialize.js. Where we will initialize the handlers after context is created. See this snippet below:
// initialize.js
...
const initializeMiddlewares = require('./middlewares/initializeMiddlewares');
const initializeHandlers = require('./handlers/initializeHandlers');
...
const initialize = () => {
...
const middlewares = initializeMiddlewares(context);
const handlers = initializeHandlers(context);
Object.assign(context, { middlewares, handlers });
return context;
}
...
Simply add the handlers after or before middlewares, then assign them back to context at the same level as middlewares.
index.js
Lastly, we get the handlers from index.js. See this snippet below:
// index.js
const initialize = require('./initialize');
const express = require('express');
const bodyParser = require('body-parser');
const app = express();
app.use(bodyParser.json());
const context = initialize();
const {
mongoDbConnector,
redisConnector,
config,
middlewares,
handlers
} = context;
const { authMiddleware } = middlewares;
const {
createUserHandler,
getUserListHandler,
getUserHandler,
updateUserHandler,
deleteUserHandler,
loginHandler
} = handlers;
app.post('/user', authMiddleware, createUserHandler);
app.get('/user', authMiddleware, getUserListHandler);
app.get('/user/:id', authMiddleware, getUserHandler);
app.patch('/user/:id', authMiddleware, updateUserHandler);
app.delete('/user/:id', authMiddleware, deleteUserHandler);
app.post('/login', loginHandler);
app.listen(...);
...This file changes the most. First, we get handlers from context right after middlewares. Then, we also destructure handlers to get createUserHandler and the rest.
Next, we remove all inline handlers from all app.post(), app.get(), etc. except app.listen(). And then set the specific handlers to each respective functions. For example, POST /user means creating a new user, so we assign createUserHandler on it.
Lastly, remove unnecessary imports such as bcrypt, authUtils, ObjectId, etc. because it is not even used anymore in index.js.
Test the code
This part of refactoring changes quite a lot, especially index.js. Your directory structure should be like this at this point:
Finally, run npm start and test every route. If nothing changes, then it works correctly.
Refactor the directories
OK, here is the last part. We already refactored much of our code. Now, we will tidy up the file locations. Let’s start with the connectors.
Connectors
Create a new directory named connectors. Then, move mongoDbConnector.js and redisConnector.js to the directory. It should look like this:
Then, create a new file inside the connectors directory named initializeConnectors.js. Just like initializeHandlers and initializeMiddlewares, initializeConnectors are for initializing connectors. Here’s a snippet of it:
// initializeConnectors.js
const MongoDbConnector = require('./mongoDbConnector');
const RedisConnector = require('./redisConnector');
const initMongoDb = (config) => {
const mongoDbConnector = new MongoDbConnector({
name: config.db.name,
host: config.db.host
});
mongoDbConnector.connect();
return mongoDbConnector;
}
const initRedis = (config) => {
const redisConnector = new RedisConnector();
redisConnector.connect(config.redis.host, config.redis.port);
return redisConnector;
}
const initializeConnectors = (config) => ({
mongoDbConnector: initMongoDb(config),
redisConnector: initRedis(config)
});
module.exports = initializeConnectors;If you look at the code closely, it’s the code we did from initialize.js. Yes, we migrate the code from initialize.js to initializeConnectors. Then we create the function initializeConnectors to create the connectors from both functions initMongoDb() and initRedis().
Next we modify initialize.js to this:
// initialize.js
const config = require('./config');
const initializeConnectors = require('./connectors/initializeConnectors');
const initializeMiddlewares = require('./middlewares/initializeMiddlewares');
const initializeHandlers = require('./handlers/initializeHandlers');
const initialize = () => {
const connectors = initializeConnectors(config);
const context = {
...connectors,
config,
collection: config.db.collections.user
};
const middlewares = initializeMiddlewares(context);
const handlers = initializeHandlers(context);
Object.assign(context, { middlewares, handlers });
return context;
}
module.exports = initialize;We remove initMongoDb(), initRedis() and imports from mongoDbConnector and redisConnector. Then, we import the recently created initializeConnectors. We call the function with config argument to get the list of connectors needed. Lastly, we set each of the connectors to context.
Utils
Next, we create another new directory named utils. This is where we put utility functions such as authUtils. Personally, I decide that a file is a utils file if it satisfies these conditions:
- It does not need initializations, unlike mongoDbConnector or redisConnector.
- It can be used anywhere.
- It is a generic type of function.
So, we move authUtils to the utils directory, and we have this final directory structure:
Conclusion
So, in this part, you should have learned all these:
- Adding password encryption in DB using
bcryptlibrary. - Verifying encrypted password using
bcryptlibrary. - Separating route
handlersfromroutes. - Creating initializers for
middlewares,handlers, andconnectors, theninitializethem onindex.js. - Restructure directories to make it easier to browse.
Phew, we covered a lot this time. It might look there are so much things to do in refactoring, but actually when you get to it, you won’t feel like it’s too much. You will think that there are some things that you are not satisfied yet and you know you can fix it and do better. And before you realize it, you will have changed much of your code.
Refactoring skills also comes with experience. The more projects you are involved in, the more you know which part of the code that can be improved based on the experience you’ve had before. You will definitely get much better.
And here’s the source code for this tutorial, as promised. Did you manage to follow the changes well?
I hope you also learned much this time. Are there some things we can do better? Let me know in the comments!
P.S. I will be on break for 2 weeks for personal retreat. Stay tuned for the next posts!
“Other than when you are very close to a deadline, however, you should not put off refactoring because you haven’t got time. Experience with several projects has shown that a bout of refactoring results in increased productivity. Not having enough time usually is a sign that you need to do some refactoring.”
― Martin Fowler, Refactoring: Improving the Design of Existing Code
