Progress and roadblocks: a journey into open-source

Enhancing generators for ES6 in the Sequelize CLI

This is the story of my discovery of ES6-class style model definitions with Sequelize (see Using ES6 classes for Sequelize 4 models). The realisation that the current Sequelize CLI model generator didn’t support it (as is expected since it’s not the most common way of doing it). Finally, the impetus to just implement it myself.

In other words, I started using a style of Sequelize models that the Sequelize CLI didn’t support. I didn’t get an answer on a “feature request” issue, so I just forked it, implemented it and published it.

The Free Open Source Software movement has its ups and downs, its hype and abandoned projects. One of the great aspects of it is that you can always (depending on the license) just fork something and publish it 🙂.

Some links:

• sequelize-cli fork repo
• fork published on npm

Live-tweet storms:

• Day 1 (struggles)
• Day 2 (fixes)
• Day 3 (first publish)
• Day 4 (finishing up)

Table of contents:

Dealing with a GitHub fork repository 🍴

1. Fork using the GitHub UI
2. Clone your fork git clone [email protected]:USERNAME/REPONAME.git

• eg. git clone [email protected]:HugoDF/sequelize-cli.git

3. Create upstream remote
4. git remote add upstream git remote add upstream https://github.com/sequelize/cli
5. (To sync with upstream) fetch upstream remote and merge it into local branch (from GitHub Help Syncing a Fork)
6. Get latest from upstream git fetch upstream
7. Merge changes git merge upstream/master
8. Send upstream changes to fork git push

Following setup instructions 📐

Getting a project set up can be daunting, luckily Sequelize CLI has some steps in CONTRIBUTING.md, they’re the following:

1. npm install

• Didn’t work on Node 10, something to do with node-gyp and SQLite

2. npm test

• takes 10-15 minutes
• failed on latest master

My goals 🛒

I wanted to:

1. Update sequelize model:create to support a second template, which uses an ES6 class model 👍 when passed a --class flag, the tasks are as follows
2. Add support for a --class flag, somewhere in model_generate.js
3. Check for args.class in model_helper.js generateFileContent, and switch templates depending on that
4. Create a class-model.js file which is a template similar to src/assets/models/model.js but uses ES6 class syntax for the Sequelize model
5. Use class-model.js instead of model.js in model_helper#generateFileContent when args.class is set (ie. true)
6. Update the migration generator (these are totally arbitrary choices but things I’ve found wanting to do in Sequelize projects):
7. use object shorthand notation for functions instead of arrow functions
8. (Ended up not doing this, because it’s trivial and nitpicky) use (sequelize, DataTypes) -> Promise function signature, ie. rename queryInterface and Sequelize to sequelize and DataTypes respectively
9. (Won’t do) Update model/index.js to a form that is more explicit and more in line with what I would use in production

• Why? This isn’t something that is compatible with current behaviour of the template, you would lose auto-loading of all models in the models folder.

Getting tests to pass 🚫

1. Failing locally…

2. Was working for https://github.com/sequelize/cli

3. Why?

   2.1/ Day 2 of trying to hack on Sequelize CLI

   I’ve got green tests… it turns out master is a fake ✅ passing build. It loads latest sequelize-cli from npm, soooo isn’t testing the code in master.

   Hard reset to latest released npm version fixed the test 👍

   — Hugo Di Francesco (@hugo__df) 31 October 2018

   Sequelize CLI Continuous Integration isn’t set up how you expect it to be set up:

• What you expect: checkout the commit’s code and test it
• What it does: fetch sequelize-cli from npm, and test that… so it’s testing the latest published release, not the latest code in the repo.

4. Tests still take a while ⏳

• they’re mostly integration tests: writing files, writing to the DB and having to setup/teardown every so often
• on my machine a bunch of tests run in 2-3 seconds each and the total time is about 10-15 minutes

Releasing the updates 🚀

Trying to contribute back upstream

A GitHub issue created on the 1st of August 2018 seems to have been lost (not even a “this isn’t something we want to do”).

It’s not the nicest experience but it’s totally fine by me, OSS maintainers don’t owe us anything after all.

This might be a niche use-case, most people seem to be happy to write models in the module.exports = () => { /* do some stuff in the closure */ return }; fashion, although looking at the number of people who read “Using ES6 classes for Sequelize 4 models”, it’s a valid way to write an application using Sequelize:

I went ahead and created a PR but I don’t expect it to get merged:

• es6 class PR on sequelize/cli repo

Publishing a fork as an npm scoped module

I always thought this is more a niche/variation type of thing, so publishing it as a fork makes sense

Update package.json to have, publishConfig is very important if you’re publishing a scoped module

{
  "name": "@hugodf/sequelize-cli",
  "publishConfig": {
    "access": "public"
  }
}

Use np 🙂:

1. npx np and go through the dialog
2. Since the tests take 10-15 minutes to run, I’ve found using the --yolo option useful

Lessons about the Open Source Movement 📖

People are paid to make mistakes, people who work for free will also make (similar) mistakes

Just like at the day job, it’s important to have empathy for “interesting things” going on in the code eg. the tests running against something other than the code.

If anything maintainers have created this project for free, in their spare time so expect more “interesting things” in the codebase.

Technical debt is a sign that the open source project has reached some level of maturity.

Remember that your situation is the following:

1. Aware of the project’s existence
2. Using the project
3. Finding a use-cases where the project doesn’t quite fit your needs
4. That use-case might not be something the project even wants to solve
5. Attempting to contribute back to the project

You’re in a very specific situation where the project has reached some level of critical mass/awareness/scale. To get here, just like at a startup attempting to scale, the maintainers have had to move relatively fast and fly by the seat of their pants, maybe put some “hacks” in some places.

No one owes you anything

As I mentioned, I opened both an issue and a PR and I don’t think either one of them is going to get a response.

That’s why I just went ahead and published the fork as a module (scoped modules is awesome for this kind of use-case 👍 npm team).

Do what you can

Every little helps:

• Mentioning something that stumped you during setup (eg. master being a fake green ✅ build)
• Documentation updates
• Opening random PRs with super-opinionated new feature 🙄
• Write blog posts documenting interesting usage of a library

Everyone is just trying to help.

Credit Ruediger Theiselmann