(Updated: )
/ #javascript #node 

ES6 by example: a module/CLI to wait for Postgres in docker-compose

When using docker-compose, it’s good practice to make anything that relies on Postgres wait for it to be up before launching. This avoids connection issues inside the app.

This post walks through how to deliver this functionality both as a CLI and a module that works both as a CommonJS module (require) and ES modules, without transpilation.

“A fast, production ready, zero-dependency ES module loader for Node 6+!” is esm’s promise. From this sample project, it’s worked.

Table of Contents

This was sent out on the Code with Hugo newsletter last Monday. Subscribe to get the latest posts right in your inbox (before anyone else).

Writing ES modules without a build step 🎨

To begin we install esm: npm install --save esm. Next we’ll need a file for our module, wait-for-pg.js:

export const DEFAULT_MAX_ATTEMPTS = 10;
export const DEFAULT_DELAY = 1000; // in ms

Trying to run this file with Node will throw:

$ node wait-for-pg.js
/wait-for-pg/wait-for-pg.js:1
export const DEFAULT_MAX_ATTEMPTS = 10;
^^^^^^

SyntaxError: Unexpected token export

export and import don’t work in Node yet (without flags), the following runs though:

$ node -r esm wait-for-pg.js

That’s if we want to run it as a script, say we want to let someone else consume it via require we’ll need an index.js with the following content:

require = require('esm')(module);
module.exports = require('./wait-for-pg');

We can now run index.js as a script:

$ node index.js

We can also require it:

$ node # start the Node REPL
> require('./index.js')
{ DEFAULT_MAX_ATTEMPTS: 10,
  DEFAULT_DELAY: 1000 }

To tell users wanting to require the package with Node, we can use the "main" field in package.json:

{
  "main": "index.js",
  "dependencies": {
    "esm": "^3.0.62"
  }
}

Sane defaults 🗃

To default databaseUrl, maxAttempts and delay, we use ES6 default parameters + parameter destructuring. Let’s have a look through some gotchas of default parameters that we’ll want to avoid:

  1. Attempting to destructure ‘null’ or ‘undefined’
  2. ‘null’ remains, undefined gets defaulted

Attempting to destructure null or undefined 0️⃣

export const DEFAULT_MAX_ATTEMPTS = 5;
export const DEFAULT_DELAY = 1000; // in ms

export function waitForPostgres({
  databaseUrl = (
    process.env.DATABASE_URL || 
    'postgres://postgres@localhost'
  ),
  maxAttempts = DEFAULT_MAX_ATTEMPTS,
  delay = DEFAULT_DELAY
}) {
  console.log(
    databaseUrl, 
    maxAttempts,
    delay
  )
}

Callings the following will throw:

$ node -r esm # node REPL
> import { waitForPostgres } from './wait-for-pg';
> waitForPostgres()
TypeError: Cannot destructure property `databaseUrl` of 'undefined' or 'null'.
    at waitForPostgres (/wait-for-pg/wait-for-pg.js:4:19)
> waitForPostgres(null)
TypeError: Cannot destructure property `databaseUrl` of 'undefined' or 'null'.
    at waitForPostgres (/wait-for-pg/wait-for-pg.js:4:19)

To avoid this, we should add = {} to default the parameter that’s being destructured (wait-for-pg.js):

export const DEFAULT_MAX_ATTEMPTS = 5;
export const DEFAULT_DELAY = 1000; // in ms

export function waitForPostgres({
  databaseUrl = (
    process.env.DATABASE_URL || 
    'postgres://postgres@localhost'
  ),
  maxAttempts = DEFAULT_MAX_ATTEMPTS,
  delay = DEFAULT_DELAY
} = {}) {
  console.log(
    databaseUrl, 
    maxAttempts,
    delay
  )
}

It now runs:

$ node -r esm # node REPL
> import { waitForPostgres } from './wait-for-pg';
> waitForPostgres()
postgres://postgres@localhost 10 1000

The values were successfully defaulted when not passed a parameter. However the following still errors:

> waitForPostgres(null)
postgres://postgres@localhost 10 1000
TypeError: Cannot destructure property `databaseUrl` of 'undefined' or 'null'.
    at waitForPostgres (/wait-for-pg/wait-for-pg.js:4:19)

‘null’ remains, undefined gets defaulted 🔎

$ node -r esm # node REPL
> import { waitForPostgres } from './wait-for-pg';
> waitForPostgres({ databaseUrl: null, maxAttempts: undefined })
null 10 1000

The values explicitly set as null doesn’t get defaulted whereas an explicit undefined and an implicit one do, that’s just how default parameters work, which isn’t exactly like the old-school way of writing this:

export const DEFAULT_MAX_ATTEMPTS = 5;
export const DEFAULT_DELAY = 1000; // in ms

export function waitForPostgres(options) {
  const databaseUrl = (
    options && options.databaseUrl ||
    process.env.DATABASE_URL ||
    'postgres://postgres@localhost'
  );
  const maxAttempts = options && options.maxAttempts || DEFAULT_MAX_ATTEMPTS;
  const delay = options && options.delay || DEFAULT_DELAY;
  console.log(
    databaseUrl, 
    maxAttempts,
    delay
  )
}

Which would yield the following:

$ node -r esm # node REPL
> import { waitForPostgres } from './wait-for-pg';
> waitForPostgres({ databaseUrl: null, maxAttempts: undefined })
'postgres://postgres@localhost' 10 1000

Since null is just as falsy as undefined 🙂 .

Waiting for Postgres with async/await 🛎

Time to implement wait-for-pg. To wait for Postgres we’ll want to:

  • try to connect to it
  • if that fails
    • try again later
  • if that succeeds
    • finish

Let’s install a Postgres client, pg using: npm install --save pg

pg has a Client object that we can pass a database URL to when instantiating it (new Client(databaseUrl)). That client instance has a .connect method that returns a Promise which resolves on connection success and rejects otherwise. That means if we mark the waitForPostgres function as async, we can await the .connect call.

When await-ing a Promise, a rejection will throw an error so we wrap all the logic in a try/catch.

  • If the client connection succeeds we flip the loop condition so that the function terminates
  • If the client connection fails
    • we increment the retries counter, if it’s above the maximum number of retries (maxAttempts), we throw which, since we’re in an async function throw is the equivalent of doing Promise.reject
    • otherwise we call another function that returns a Promise (timeout) which allows us to wait before doing another iteration of the loop body
  • We make sure to export function waitForPostgres() {}

wait-for-pg.js:

import { Client } from 'pg';

export const DEFAULT_MAX_ATTEMPTS = 10;
export const DEFAULT_DELAY = 1000; // in ms

const timeout = ms => new Promise(
  resolve => setTimeout(resolve, ms)
);

export async function waitForPostgres({
  databaseUrl = (
    process.env.DATABASE_URL || 
    'postgres://postgres@localhost'
  ),
  maxAttempts = DEFAULT_MAX_ATTEMPTS,
  delay = DEFAULT_DELAY
} = {}) {
  let didConnect = false;
  let retries = 0;
  while (!didConnect) {
    try {
      const client = new Client(databaseUrl);
      await client.connect();
      console.log('Postgres is up');
      client.end();
      didConnect = true;
    } catch (error) {
      retries++;
      if (retries > maxAttempts) {
        throw error;
      }
      console.log('Postgres is unavailable - sleeping');
      await timeout(delay);
    }
  }
}

Integrating as a CLI with meow 😼

meow is a CLI app helper from Sindre Sohrus, install it: npm install --save meow Create wait-for-pg-cli.module.js:

import {
  waitForPostgres,
  DEFAULT_MAX_ATTEMPTS,
  DEFAULT_DELAY
} from './wait-for-pg';
import meow from 'meow';

const cli = meow(`
    Usage
      $ wait-for-pg <DATABASE_URL>
    Options
      --max-attempts, -c Maximum number of attempts, default: ${DEFAULT_MAX_ATTEMPTS}
      --delay, -d Delay between connection attempts in ms, default: ${DEFAULT_DELAY}
    Examples
      $ wait-for-pg postgres://postgres@localhost:5432 -c 5 -d 3000 && npm start
      # waits for postgres, 5 attempts at a 3s interval, if
      # postgres becomes available, run 'npm start'
`, {
    inferType: true,
    flags: {
      maxAttempts: {
        type: 'string',
        alias: 'c'
      },
      delay: {
        type: 'string',
        alias: 'd'
      }
    }
  });
console.log(cli.input, cli.flags);

We use inferType so that the values for maxAttempts and delay get converted to numbers instead of being strings. We can run it using:

$ node -r esm wait-for-pg-cli.module.js
[] {}

The following is a template string, it will replace things inside of ${} with the value in the corresponding expression (in this instance the value of the DEFAULT_MAX_ATTEMPTS and DEFAULT_DELAY variables)

`
  Usage
    $ wait-for-pg <DATABASE_URL>
  Options
    --max-attempts, -c Maximum number of attempts, default: ${DEFAULT_MAX_ATTEMPTS}
    --delay, -d Delay between connection attempts in ms, default: ${DEFAULT_DELAY}
  Examples
    $ wait-for-pg postgres://postgres@localhost:5432 -c 5 -d 3000 && npm start
    # waits for postgres, 5 attempts at a 3s interval, if
    # postgres becomes available, run 'npm start'
`;

To get the flags and first input, wait-for-pg-cli.module.js:

import {
  waitForPostgres,
  DEFAULT_MAX_ATTEMPTS,
  DEFAULT_DELAY
} from './wait-for-pg';
import meow from 'meow';

const cli = meow(`
    Usage
      $ wait-for-pg <DATABASE_URL>
    Options
      --max-attempts, -c Maximum number of attempts, default: ${DEFAULT_MAX_ATTEMPTS}
      --delay, -d Delay between connection attempts in ms, default: ${DEFAULT_DELAY}
    Examples
      $ wait-for-pg postgres://postgres@localhost:5432 -c 5 -d 3000 && npm start
      # waits for postgres, 5 attempts at a 3s interval, if
      # postgres becomes available, run 'npm start'
`, {
    inferType: true,
    flags: {
      maxAttempts: {
        type: 'string',
        alias: 'c'
      },
      delay: {
        type: 'string',
        alias: 'd'
      }
    }
  });
waitForPostgres({
  databaseUrl: cli.input[0],
  maxAttempts: cli.flags.maxAttempts,
  delay: cli.flags.delay,
}).then(
  () => process.exit(0)
).catch(
  () => process.exit(1)
);

If you don’t have a Postgres instance running on localhost the following shouldn’t print Here, that’s thanks to process.exit(1) in the .catch block:

$ node -r esm wait-for-pg-cli.module.js -c 5 && echo "Here"
Postgres is unavailable - sleeping
Postgres is unavailable - sleeping
Postgres is unavailable - sleeping
Postgres is unavailable - sleeping
Postgres is unavailable - sleeping

Packaging and clean up 📤

We can use the "bin" key in package.json to be able to run the command easily:

{
  "main": "index.js",
  "bin": {
    "wait-for-pg": "./wait-for-pg-cli.js"
  },
  "dependencies": {
    "esm": "^3.0.62",
    "meow": "^5.0.0",
    "pg": "^7.4.3"
  }
}

Where wait-for-pg-cli.js is:

#!/usr/bin/env node
require = require("esm")(module/*, options*/);
module.exports = require('./wait-for-pg-cli.module');

Don’t forget to run chmod +x wait-for-pg-cli.js esm allows us to use top-level await, that means in wait-for-pg-cli.module.js, we can replace:

waitForPostgres({
  databaseUrl: cli.input[0],
  maxAttempts: cli.flags.maxAttempts,
  delay: cli.flags.delay,
}).then(
  () => process.exit(0)
).catch(
  () => process.exit(1)
);

With:

try {
  await waitForPostgres({
    databaseUrl: cli.input[0],
    maxAttempts: cli.flags.maxAttempts,
    delay: cli.flags.delay,
  });
  process.exit(0);
} catch (error) {
  process.exit(1);
}

Running the CLI throws:

$ ./wait-for-pg-cli.js
wait-for-pg/wait-for-pg-cli.module.js:36
  await waitForPostgres({
  ^^^^^

SyntaxError: await is only valid in async function

We need to add "esm" with "await": true in package.json:

{
  "main": "index.js",
  "bin": {
    "wait-for-pg": "./wait-for-pg-cli.js"
  },
  "dependencies": {
    "esm": "^3.0.62",
    "meow": "^5.0.0",
    "pg": "^7.4.3"
  },
  "esm": {
    "await": true
  }
}

This now works:

$ ./wait-for-pg-cli.js -c 1
Postgres is unavailable - sleeping

Extras

This was sent out on the Code with Hugo newsletter last Monday. Subscribe to get the latest posts right in your inbox (before anyone else).

Publishing to npm with np

  1. Run: npm install --save-dev np
  2. Make sure you have a valid "name" field in package.json, eg. "@hugodf/wait-for-pg"
  3. npx np for npm v5+ or ./node_modules/.bin/np (npm v4 and down)

Pointing to the ESM version of the module

Use the "module" fields in package.json

{
  "name": "wait-for-pg",
  "version": "1.0.0",
  "description": "Wait for postgres",
  "main": "index.js",
  "module": "wait-for-pg.js",
  "bin": {
    "wait-for-pg": "./wait-for-pg-cli.js"
  },
  "dependencies": {
    "esm": "^3.0.62",
    "meow": "^5.0.0",
    "pg": "^7.4.3"
  },
  "devDependencies": {
    "np": "^3.0.4"
  },
  "esm": {
    "await": true
  }
}

A Promise wait-for-pg implementation

import { Client } from 'pg';

export const DEFAULT_MAX_ATTEMPTS = 10;
export const DEFAULT_DELAY = 1000; // in ms

const timeout = ms => new Promise(
  resolve => setTimeout(resolve, ms)
);

export function waitForPostgres({
  databaseUrl = (
    process.env.DATABASE_URL ||
    'postgres://postgres@localhost'
  ),
  maxAttempts = DEFAULT_MAX_ATTEMPTS,
  delay = DEFAULT_DELAY,
} = {},
  retries = 1
) {
  const client = new Client(databaseUrl);
  return client.connect().then(
    () => {
      console.log('Postgres is up');
      return client.end();
    },
    () => {
      if (retries > maxAttempts) {
        return Promise.reject(error);
      }
      console.log('Postgres is unavailable - sleeping');
      return timeout(delay).then(
        () => waitForPostgres(
          { databaseUrl, maxAttempts, delay },
          retries + 1
        )
      );
    }
  );
}

unsplash-logoMatthew Henry

Author

Hugo Di Francesco

Co-author of "Professional JavaScript", "Front-End Development Projects with Vue.js" with Packt, "The Jest Handbook" (self-published). Hugo runs the Code with Hugo website helping over 100,000 developers every month and holds an MEng in Mathematical Computation from University College London (UCL). He has used JavaScript extensively to create scalable and performant platforms at companies such as Canon, Elsevier and (currently) Eurostar.

Get The Jest Handbook (100 pages)

Take your JavaScript testing to the next level by learning the ins and outs of Jest, the top JavaScript testing library.

Find out more close