Summary

• Asynchronous request batching
• Endpoint creation
• Benchmarks
• Optimizing the api
• Benchmarks after optimization

Asynchronous request batching

Is an optimization technique consisting from 6 steps as shown on the diagram bellow.

1. We have a client which requests an api
2. The request is forwarded by node.js to the database or other data storage service.
3. Database is handling the request and then it is sending the response to node.js
4. For every other client which is requesting the same api in time of the database process, we register a handler
5. We store the request handler in a batching queue.
6. After we got the response from the database we iterate the queue and send back the response to all clients.

Endpoint creation

We will build a simple endpoint using express and mongodb.

// index.js file
const express = require('express');
const { MongoClient } = require('mongodb');
const assert = require('assert');

const app = express();

MongoClient.connect('mongodb://localhost:27017', {}, (err, client) => {
  const wormsRepository = require('./worms.repository')(client);
  assert.equal(err, null);

  app.get('/api/v1/worms/:class', (req, res) => {
    wormsRepository.findByClass(req.params.class, (err, worms) => {
      return res.json(worms);
    });
  });

  app.listen(8080, (err) => {
    assert.equal(err, null);
    console.log('Server is running on: http://localhost:8080/');
  });
});

The source code of the repository file:

// worms.repository.js

module.exports = (client) => {
  const db = client.db('wormery');
  const col = db.collection('worms');

  function findByClass(className, cb) {
    return col.find({ class: className }, (err, worms) => {
      if (err) return cb(err);

      return worms
        .limit(10)
        .toArray(cb);
    });
  }

  return {
    findByClass,
  };
};

I added in a mongodb database 1,000,000 documents.

Benchmarks

I used the apache benchmark tool to test the performance:
ab -c 300 -t 10 http://localhost:8080/api/v1/worms/utilize

utilize is the class name generated by faker, it is just a random word.

Server Software:
Server Hostname:        localhost
Server Port:            8080

Document Path:          /api/v1/worms/utilize
Document Length:        1131 bytes

Concurrency Level:      300
Time taken for tests:   10.001 seconds
Complete requests:      2977
Failed requests:        0
Total transferred:      3992157 bytes
HTML transferred:       3366987 bytes
Requests per second:    297.67 [#/sec] (mean)
Time per request:       1007.841 [ms] (mean)
Time per request:       3.359 [ms] (mean, across all concurrent requests)
Transfer rate:          389.81 [Kbytes/sec] received

For 10 seconds with a 300 concurrent users I managed to handle 2977 requests, a very poor result.

Optimizing the api

We will create a new file named worms-batching.repository.js; Where we will optimize our code using the asynchronous batching technique.

const activeQueues = {};

module.exports = (client) => {
  const wormsRepository = require('./worms.repository')(client);

  function findByClass(term, cb) {
    if (activeQueues[term]) {
      return queues[term].push(cb);
    }

    activeQueues[term] = [cb];

    return wormsRepository.findByClass(term, (err, worms) => {
      const queue = activeQueues[term];
      activeQueues[term] = null;
      queue.forEach(callback => callback(err, worms));
    });
  }

  return {
    findByClass,
  };
};

As we can observe we store all the requests in an activeQueue and when we get the result from database we just iterate the queue and send the responses and then we nullify the existing queue.

Benchmarks after optimization

Server Software:
Server Hostname:        localhost
Server Port:            8080

Document Path:          /api/v1/worms/utilize
Document Length:        1131 bytes

Concurrency Level:      300
Time taken for tests:   10.007 seconds
Complete requests:      33339
Failed requests:        0
Total transferred:      44707599 bytes
HTML transferred:       37706409 bytes
Requests per second:    3331.63 [#/sec] (mean)
Time per request:       90.046 [ms] (mean)
Time per request:       0.300 [ms] (mean, across all concurrent requests)
Transfer rate:          4363.01 [Kbytes/sec] received

After optimization we managed to serve 33339 requests. 11 times better than the initial response.