Jazelle

Incremental, cacheable builds for large Javascript monorepos. Uses Bazel

Getting started
Reference
- CLI
- API
- Configuration
- Bazel rules
Misc

Why use Jazelle

Jazelle is designed for large organizations where different teams own different projects within a monorepo, and where projects depend on compiled assets from other projects in the monorepo. In terms of developer experience, it's meant to be a low-impact drop-in replacement for common day-to-day web stack commands such as yarn add, yarn build and yarn test.

Jazelle leverages Bazel for incremental/cacheable builds and should be able to integrate with Bazel rules from non-JS stacks. This is helpful if the rest of your organization is also adopting Bazel, especially if others in your organization are already investing into advanced Bazel features such as distributed caching. Jazelle can also be suitable if you develop libraries and want to test for regressions in downstream projects as part of your regular development workflow.

Due to its integration w/ Bazel, Jazelle can be a suitable solution if long CI times are a problem caused by running too many tests.

Jazelle can also be a suitable solution if the frequency of commits affecting a global lockfile impacts developer velocity.

If you just have a library of decoupled components, Jazelle might be overkill. In those cases, you could probably get away with using a simpler solution, such as Yarn workspaces, Lerna or Rush.

Scaffold a workspace

mkdir my-monorepo
cd my-monorepo
jazelle init

The jazelle init command generates Bazel WORKSPACE, BUILD.bazel and .bazelversion files, along with the Jazelle configuration file manifest.json. If you are setting up Jazelle on an existing Bazel workspace, see Bazel rules.

Configure Bazel rules

Check that the .bazelversion file at the root of your repo contains your desired Bazel version. For example:

5.1.0

Check that the WORKSPACE file at the root of your repo is using the desired versions of Jazelle, Node and Yarn:

load("@bazel_tools//tools/build_defs/repo:http.bzl", "http_archive")
http_archive(
  name = "jazelle",
  url = "https://registry.yarnpkg.com/jazelle/-/jazelle-[version].tgz",
  sha256 = "SHA 256 goes here",
  strip_prefix = "package",
)

load("@jazelle//:workspace-rules.bzl", "jazelle_dependencies")
jazelle_dependencies(
  node_version = "16.15.0",
  node_sha256 = {
    "darwin-x64": "a6bb12bbf979d32137598e49d56d61bcddf8a8596c3442b44a9b3ace58dd4de8",
    "linux-x64": "ebdf4dc9d992d19631f0931cca2fc33c6d0d382543639bc6560d31d5060a8372",
    "win-x64": "dbe04e92b264468f2e4911bc901ed5bfbec35e0b27b24f0d29eff4c25e428604",
    "darwin-arm64": "ad8d8fc5330ef47788f509c2af398c8060bb59acbe914070d0df684cd2d8d39b",
    "linux-arm64": "b4080b86562c5397f32da7a0723b95b1df523cab4c757688a184e3f733a7df56",
  },
  yarn_version = "1.19.1",
  yarn_sha256 = "fdbc534294caef9cc0d7384fb579ec758da7fc033392ce54e0e8268e4db24baf",
)

Jazelle SHA256 checksum can be computed through the following command:

curl -fLs https://registry.yarnpkg.com/jazelle/-/jazelle-[version].tgz | openssl sha256

Node SHA256 checksums can be found at https://nodejs.org/dist/v[version]/SHASUMS256.txt. Use the checksums for these files:

darwin-x64: node-v[version]-darwin-x64.tar.gz
linux-x64: node-v[version]-linux-x64.tar.xz
win-x64: node-v[version]-win-x64.zip
darwin-arm64: node-v[version]-darwin-arm64.tar.gz
linux-arm64: node-v[version]-linux-arm64.tar.xz

Yarn SHA256 checksum can be computed through the following command:

curl -fLs https://github.com/yarnpkg/yarn/releases/download/v[version]/yarn-[version].js | openssl sha256

Double check that the BUILD.bazel at the root of your repo contains this code:

load("@jazelle//:build-rules.bzl", "jazelle")

jazelle(name = "jazelle", manifest = "manifest.json")

package.json file

There should be a package.json at the root of the monorepo, with a workspaces field:

{
  "workspaces": [
    "path/to/project-1",
    "path/to/project-2"
  ]
}

The workspaces field in this file should list every project that you want Jazelle to manage.

Setup .gitignore

Add the following entries to .gitignore

third_party/jazelle/temp
bazel-*

What to commit to version control

DO commit

package.json file
manifest.json file
WORKSPACE file
BUILD.bazel files
.bazelversion file
.bazelignore file
third_party/jazelle/BUILD.bazel file
third_party/jazelle/scripts folder
root yarn.lock file

DO NOT commit

/third_party/jazelle/temp folder
node_modules folders
bazel-[*] folders

Typical usage

CLI installation
- Upgrading
Onboarding a project
- Troubleshooting a failed onboarding
Day-to-day usage
Using Bazel
Getting out of bad states

CLI installation

Install the CLI globally:

# install
yarn global add jazelle

# verify it's installed
jazelle version

If the repo is already scaffolded, you can use the script in it:

third_party/jazelle/scripts/install-run-jazelle.sh version

Upgrading

yarn global upgrade jazelle

If upgrading fails, it's probably because you didn't follow the installation instructions. In that case, try reinstalling:

npm uninstall jazelle --global
yarn global remove jazelle
yarn global add jazelle

It's ok for users to have different versions of Jazelle installed. Jazelle runs all commands via Bazelisk, which enforces that the Bazel version specified in .bazelversion is used. Bazel, in turn, enforces that the Node and Yarn versions specified in WORKSPACE are used.

Onboarding a project

Copy and paste your project into the monorepo, at a desired path.
Open the root package.json file and add the path to your project in workspaces.
Ensure your project's package.json has scripts fields called build, test, lint and flow.
Optionally, verify that your dependency versions match the dependency versions used by other projects in the monorepo. To verify, run jazelle check. To upgrade a dependency, run jazelle upgrade [the-dependency] to get the latest or jazelle upgrade [the-dependency]@[version] from your project folder.
Run jazelle install from your project folder to generate Bazel BUILD files, and install dependencies. This may take a few minutes the first time around since Bazel needs to install itself and its dependencies. Subsequent calls to jazelle install will be faster.
Run jazelle test to verify that your project builds and tests pass. Optionally run jazelle run lint and jazelle run flow.

Troubleshooting a failed onboarding

If building your project fails, open the BUILD.bazel files and double check that the dist argument in the web_library call points to the folder where you expect compiled assets to be placed. This folder is often called dist or bin. Note that BUILD.bazel files may also be created in dependencies' folders, if they did not already have them. Use version control to identify where newly generated BUILD.bazel files were created and review their dist arguments.

EPERM

If you get permission errors (EPERM), it's likely because the Bazel sandbox disables write permissions on input files and there are compiled assets in your source code tree that are being picked up by the glob call in the BUILD.bazel file.

Delete the compiled assets that are generated by your NPM build script. You could also use the exclude argument of the glob in your BUILD.bazel file to help team members avoid the pitfall.

web_library(
    name = "library",
    deps = [
        "//third_party/jazelle:node_modules",
    ],
    srcs = glob(["**/*"], exclude = ["dist/**"]),
)

Module not found

This error happens if running an app and Node is unable to find the dependency when requireing it. It can also happen if static analysis tooling depends on build output of dependencies and you use a command that bypasses Bazel.

Check that the module is actually declared in package.json.
Check if the module is a peer dependency. If so, ensure it's also a devDependency (or a regular dependency).
Try jazelle purge && jazelle install from your project folder.
Ensure that your NPM build script does not run other tools (e.g. lint).
If you ran a command that is not documented in bazel --help (e.g. jazelle lint), try running the Bazel-enabled equivalent (jazelle run lint) instead.

Script must exist

If you get an error saying a script must exist, make sure your project has the relevant NPM script. For example, if you ran jazelle build, make sure your package.json has a scripts.build field. If it doesn't need to have one, simply create one with an empty value. If you do have that field, one of your project's local dependencies may be missing it.

Day-to-day usage

Navigate to a project in the monorepo, then use CLI commands, similar to how you would with yarn

# navigate to your project folder
cd path/to/project-1

# generates Bazel build files for relevant projects, if needed
jazelle install

# start project in dev mode
jazelle run

# run tests
jazelle test

# lint
jazelle run lint

# type check
jazelle run flow

# add dependency
jazelle add react@16.8.2

Using Bazel

Jazelle provides six build rules: jazelle, web_library, web_binary, web_executable, web_test and flow_test.

jazelle allows you to run Jazelle as a Bazel target (e.g. if you have Bazel installed globally, but not Jazelle)
web_library defines what source files and dependencies comprise a project. The jazelle install command automatically generates a web_library() declaration with the correct list of dependencies (by looking into the project's package.json)
web_binary builds a project and runs a project
web_executable runs a project (without building)
web_test runs a test script for a project
flow_test type checks the project

If you add or remove an entry in your package.json that points to a local project, Jazelle updates the yarn.lock file and adds the dependency to the deps field of the web_library declation in your project's BUILD.bazel file. In Bazel, dependencies are declared using label syntax. A label consists of a // followed by the path to the project, followed by a : followed by the name field of the web_library declaration of the project.

For example, if you have a project in folder path/to/my-project whose web_library has name = "hello", then its label is //path/to/my-project:hello.

# an example BUILD.bazel file for a project with a dependency
web_library(
  name = "my-project",
  deps = [
    # depend on a project that lives under ./my-other-project
    "//my-other-project:my-other-project",
  ],
  srcs = glob(["**/*"]),
  dist = "dist",
)

Getting out of bad states

While Jazelle attempts to always keep the workspace in a good state, it may be possible to get into a corrupt state, for example, if you manually edit system files (such as generated files in the /third_party/jazelle/temp folder).

Another way to get into a bad state is to change the name of a project. Currently, Jazelle does not support re-syncing depenency graphs after project name changes, since this use case is rare and the required checks would slow down CLI commands.

If you get into a bad state, here are some things you can try: