Releasing CLI tools

Photo by Dan Cristian Pădureț on Unsplash

Many developers, at least once, create a CLI tool. It's a fascinating and satisfying process. The CLI tool is the type of product any developer can make in one day. It does not require infrastructure to host a service or an excellent user interface with intricate interactions like on the web. It's just the language of choice and the task a program should do.

Writing tools for personal use is easy. Sometimes it's not even necessary to compile a program. For example, NodeJS applications can be launched using a simple command like this: $ node app.js --some params. The situation changes when we want to share our application. Sharing is beneficial for everyone. It helps community members — everyone can enjoy the service your application delivers. And the community helps to improve the program by fixing bugs, enhancing technical quality, and adding new features.

Working on a product with a team differs from solo coding. Lots of processes require automation and standardisation. The distribution and maintenance of the product are different as well.

Here, I am sharing how I configure a development and release process for a CLI tool project — from CI/CD workflow configurations to bug tracking in released binaries. To support the article with examples, I will refer to outdated-metrics and snail-release, the repositories I recently worked on. Even though these products are written for different runtimes (NodeJS and Deno), the development and release processes share a lot.

CI/CD

CI/CD is the most crucial part of the software development process that is often neglected. However, it silently wastes lots of developers' time when implemented poorly. Therefore, keeping CI/CD workflows lean, fast, and up-to-date is essential to minimise maintenance efforts, have a good development pace, and maximise time for product development.

CI/CD has never been easier than now. With the introduction of GitHub actions, the community got access to a powerful tool that automates many aspects of the software development cycle.

GitHub actions provide a unified and simple syntax to configure code processing activities, such as testing and building. GitHub actions encapsulate the external service calls or, in some cases, completely replace them (for example, code quality analysis, code coverage, etc.)

I'm using GitHub actions for all my projects. Therefore, this article's CI/CD workflow examples are the GitHub action examples. Please share your workflow configurations if you use another CI/CD in the comments. It's always great to learn something new.

Build, test, and lint

Build, test and lint are must-have jobs for any software product regardless of language or runtime platform.

Every language and runtime has tools to build, test, lint, and format the code. These tools can be embedded into a language/runtime, like in Deno or Go, or be a part of an ecosystem, like NodeJS and a variety of tools available via NPM.

GitHub actions support many runtimes and languages. I use setup-deno action by denoland and deno tools like fmt, lint, test and compile for Deno projects. An example of test and build workflow is available in the snail-release repository.

For NodeJS projects, I use setup-node action and tools like a jest, ts-standard, eslint and pkg. An example of test and build workflow is available in the outdated-metrics repository.

Dependencies management

Another critical part of the software development process is dependencies management. Today, most software products rely on third parties libraries and tools. Every dependency is upgraded separately, and dependent products should be updated timely. Delaying dependency upgrades can cause security and reliability issues and, in some cases, significant code changes.

Dependencies management should be a part of the CI/CD pipeline. GitHub provides a Dependabot service that automates dependencies updates and is easy to add to a project. To enable dependabot for a GitHub project, a configuration should be saved to .github/dependabot.yml. Following is an example of the dependabot config for a NodeJS project.

Snippet 1. Dependabot configuration sample

Versioning and release

Any software product is released in steps. Initial minimal product, new features and bug fixes are the stages of the software product evolution. It is constantly changing. When maintaining and using software products, it is essential to have the ability to differentiate the various versions of the product.

Semantic versioning provides a set of rules to generate software product versions. Semantic-release automates the version generation and package management processes. Its functionality can be extended with plugins.

Using semantic-release

The semantic-release relies on the commit messages format. It analyses the Git commit history to generate a new version. It uses Angular commit message format but can be altered. After the commit history is analysed, the next version is calculated and used in the release process.

Figure 1. Semantic-release simple workflow

An important detail about semantic-release is that it does not build binaries or other release artifacts. It is done separately.

In Figure 1, the final step of the workflow — 3. Release new version — depends on plugins and their configurations. There are default plugins available. If it's not enough, additional plugins can be installed.

Semantic-release as a version generator

Depending on your needs, you can entirely rely on the semantic-release tool or use it as a version generator and release artifacts separately.

• Use semantic-release to generate versions and publish artifacts. Suitable for packages and library release.
• Use semantic-release as a version generator. Suitable when you need to get the next version independently of artifacts processing. In this case, only steps — 1. Analyse Git commit history, and 2. Generate new version — are used. The artifacts processing is implemented separately, for example, with the help of action-gh-release. This approach suits better when releasing binaries or services.

Release configuration

There are two essential configuration pieces for the release:

• Semantic-release configuration — a .releaserc file stored at the root of the project
• GitHub action release workflow, stored in .github/workflows

The release configuration defines the behaviour of the semantic-release tool and its plugins.

Snippet 2. Semantic-release configuration

The configuration above tells semantic-release to run only three processes:

• generate the next version by using semantic-release/commit-analyser
• generate release notes by using semantic-release/release-notes-generator
• create a Git tag and publish a new release to GitHub by using semantic-release/github

It also has a customised version of the semantic-release/github plugin configuration. Disabled successComment and failComment prevent semantic-release from fetching GitHub issues and trying to update them (it causes GitHub API rate limiting on free accounts). Assets configuration is a relative path of where the release artifacts are. Semantic-release takes artifacts in the assets path and adds them to the GitHub release. The assets path should match the output directory of the build task.

The following example is a GitHub action workflow that orchestrates the release process.

Snippet 3. Release workflow sample

This is an example of a CLI tool release workflow. The semantic-release is used to generate a version and publish the GitHub release. The intention is to create a new version, build binaries and upload them to the GitHub release page. The two jobs — build and release — are created to implement the workflow.

The build job produces artifacts and stores them into dist directory.

The release job needs artifacts produced by the build job. needs: build configuration is used to pause the release job until build artifacts are available. Then it invokes semantic-release to create a new release in GitHub. The artifacts from dist directory are added to the GitHub release.

To run a workflow manually, the on: workflow_dispatch configuration is used.

Formatting commit messages

Commit message format is essential for the generation of the correct version. To minimise formatting errors, the message generation process can be automated. To do this, you need the following:

• @commitlint/config-conventional — a set of message format rules
• @commitlint/cli — a tool to verify commit messages according to the format rules
• commitizen — a CLI tool that guides through the commit message generation process
• husky — Git hooks extension that can invoke commitizen and verify message before it's committed to Git

Such configuration provides a robust way to control the commit messages format. But it adds lots of friction to the process, especially when new changes are added via Pull Request and the Squash and Merge technique is used. In this case, only the Pull Request commit message should have a valid format.

Most of the time, I work with feature branches and use pull request messages linter. I don't control the formatting of commit messages of feature branches. In case when I need to commit directly to the main branch, I manually format the commit message. While this approach works well for individual contributors, the granular commit message formatting might be a better strategy for a team or when most changes are merged directly to the main trunk.

Figure 2. Granular commit message formatting vs PR message formatting

Distribution

When the release is finished, the tool is ready to be distributed to users. There are a few ways to do it.

Binaries

The release workflow sample in Snippet 3. builds CLI tool binaries and uploads them to the GitHub release page. Users can find and download binaries for their platform from the GitHub release page.

Homebrew

Another way to get the tool is to install it via homebrew. Homebrew is a package manager for macOS and Linux. It consists of the CLI tool brew (the package manager tool) and supporting repositories.

When distributing a tool via homebrew, the brew needs to know where to take the tool's binaries and how to verify them. This information is provided in the package definition file called formula.

Formulas are organised into taps — a directory containing formulas (usually a Git repository). Brew uses taps to search, install and update formulas.

A global/default tap stores formulas for the most popular tools. You can open a Pull Request to add a formula for your tool to the global tap. Alternatively, you can create your tap and register it by using brew tap command.

There's a convention for naming taps based on the GitHub repository. The repository should have a homebrew- prefix. In this case, to add a new tap, the command looks like brew tap awesome/tool , and the formula repository is https://github.com/awesome/homebrew-tool. If the convention is not followed or the tap is based on a non-GitHub repository, the repository URL is required brew tap awesome/tool https://some.other.repository.

Formulas are written in Ruby and describe how to install the tool. The syntax allows for providing instructions for various operating systems and architectures.

The formula file name should match the class name. In the following example, the class name is SuperCliTool, meaning it should be stored in super-cli-tool.rb file.

Snippet 4. Homebrew formula sample

A few things are happening in the formula:

• It provides the tool's description, homepage and the latest version
• For each OS and architecture, it provides a specific URL for downloading the tool's binaries
• For each OS and architecture, it provides binaries' checksums for verification
• In this particular example, the binaries' suffixes contain the OS and platform names. The install function (def install) is used to rename binaries.
• Finally, a simple test is run to ensure the binaries are successfully installed and usable.

In homebrew-snail-release, you can find a prototype for the formula sample. The binaries' checksums in the formula match the values from checksums.txt of the snail-release's latest release.

Beyond release — monitoring and error tracking

After the application is released, the maintenance and support phase starts. It's essential to know about any issues with the application. For this goal, monitoring and error-tracking tools are used, for example, sentry and bugsnag.

The following snippet shows how to add sentry to a Deno project.

Snippet 5. Using sentry in a Deno project

You also need to add filesystem and network permissions to let sentry SDK collect and send information about issues.

Environment variables or CLI arguments are an excellent way to configure the code running on a server. It's more challenging to add configurations to binaries. For example, Go uses linker flags (ldflags) to insert dynamic information into the binary at build time. But Deno does not provide this option, and it's up to a developer to find a solution.

Thanks for reading. Please 👏 if you liked the article. And share your release workflows in the comments.