Bazel Remote Execution

Namespace runs your Bazel actions on workers it keeps running for your workspace, so a build can spread across many workers without waiting for them to start. Actions can also run on a platform other than the one driving Bazel, which is how a Linux host can build and test macOS artifacts.

Remote Execution builds on the Namespace Bazel cache. The configuration written by nsc bazel setup includes both the remote executor and the remote cache, so remotely executed actions share artifacts with your CI and local builds.

Getting started

Remote Execution is enabled for every workspace. Use nsc bazel setup to create a Bazel configuration file for the execution cluster:

$
nsc bazel setup --bazelrc=~/.namespace.bazelrc

$
bazel --bazelrc=~/.namespace.bazelrc build //...

How it works

A Bazel Remote Execution cluster has three components:

• Scheduler: accepts Bazel Remote Execution API requests and assigns actions to workers.
• Storage: stores content-addressable storage (CAS) entries durably across the cluster.
• Workers: execute Bazel actions.

Each worker advertises a fixed number of execution slots. A slot represents one unit of concurrent work, so a worker with four slots can run up to four actions at the same time. Namespace adds workers as a build requires more capacity.

Example: Linux host to Linux workers

This example runs Bazel from a Linux host and executes actions on Linux workers. It uses a public Bazel repository so you can try the flow without changing your own project first.

$ git clone https://github.com/bazelbuild/examples.git
$ cd examples/cpp-tutorial/stage3
 
$ nsc bazel setup --bazelrc=~/.namespace.bazelrc --key=examples-linux
$ bazel --bazelrc=~/.namespace.bazelrc build //...

Actions without explicit platform properties are routed to Linux workers.

Example: Linux host to macOS workers

Bazel can also route selected actions to macOS workers from a Linux client. The linux-to-mac-rbe-demo repository demonstrates that setup.

This example requires macos/arm64 workers to be enabled for your workspace. If you see an error such as no on-demand worker capacity configured for platform "macos/arm64", reach out to the Namespace team to enable macOS worker capacity.

$ git clone https://github.com/namespacelabs/linux-to-mac-rbe-demo.git
$ cd linux-to-mac-rbe-demo
 
$ nsc bazel setup --bazelrc=~/.namespace.bazelrc --key=linux-to-mac-demo
$ bazelisk --bazelrc=~/.namespace.bazelrc build //DemoApp:DemoApp

For your own targets, route an action to macOS by setting Bazel execution properties on that target or platform, for example:

exec_properties = {
    "OSFamily": "macos",
    "Arch": "arm64",
}

Actions without macOS execution properties continue to run on the default Linux workers, allowing one Bazel invocation to use both platforms.

Remote Execution from a Devbox

You can enable Bazel Remote Execution on any Devbox, including one that already exists. Run setup from inside the Devbox, whether or not it was created with Bazel caching enabled.

$
devbox create --checkout=github.com/your-org/your-repo

$
devbox ssh my-devbox

$
nsc bazel setup --bazelrc=/home/devbox/.bazelrc

$
echo $BAZELRC

$
echo 'export BAZELRC="/home/devbox/.bazelrc"' >> ~/.bashrc

$
bazel build //...

Keep repository-specific execution settings in a bazelrc checked into your repository rather than editing the generated one, since setup overwrites it each time it runs. Combine the two with repeated --bazelrc flags, using the BAZELRC environment variable that the Devbox sets to the generated file:

$
bazel --bazelrc=$BAZELRC --bazelrc=tools/bazelrc/rbe.bazelrc build //...

Remote Execution from CI

By default, nsc bazel setup uses your interactive login and writes a bazelrc that authenticates to the cluster with a short-lived mTLS client certificate. For CI/CD pipelines, automation, or any environment without an interactive login, you can instead authenticate with a revocable token.

Revocable tokens are long-lived but can be revoked at any time from cloud.namespace.so/user/sessions or with nsc token revoke.

$ nsc bazel create-token \
  --token rbe-token.json \
  --expires_in 10d

$ nsc bazel setup --token rbe-token.json --bazelrc=namespace.bazelrc

$ bazel --bazelrc=namespace.bazelrc build //...

Additional configuration options

Bazel execution properties can reserve worker CPU capacity and select the isolation mode on Linux workers:

exec_properties = {
    "cpu": "2",
    "namespace_action_isolation": "sandboxed",
    "namespace_requires_network": "true",
}

Use --remote_default_exec_properties to apply a property to actions that do not override it:

$ bazel --bazelrc=~/.namespace.bazelrc build //... \
  --remote_default_exec_properties=cpu=2 \
  --remote_default_exec_properties=namespace_action_isolation=sandboxed \

Reserve CPU capacity

The cpu property tells the scheduler how much CPU capacity to reserve while an action runs. Values can be whole or fractional numbers, such as "2" or "0.5". The scheduler considers both execution slots and CPU reservations when assigning concurrent actions to a worker.

The reservation is a scheduling input, not a CPU limit applied to the action. An omitted or zero value does not reserve CPU capacity beyond the action's execution slot.

resources:cpu is accepted as an alternative property name.

Select action isolation

The namespace_action_isolation property controls how a Linux worker runs an action:

The workspace default is none unless configured otherwise. Sandboxed isolation is supported only on Linux workers at the moment. Contact support@namespace.so to change the setting.

Sandboxed actions receive an isolated, loopback-only network namespace by default. Set namespace_requires_network to true on an action that needs network access. The action keeps its filesystem, process, mount, UTS, and IPC isolation, but shares the worker's network namespace. Omit the property for actions that do not need network access.

Custom worker pools

Named worker pools let actions select a worker machine type, execution slot count, and custom base image. Namespace creates a pool when its first action is queued, starts workers for it on demand, and routes only actions that name that pool to those workers. Actions without a pool use the workspace's default workers.

To use custom worker pools, define each pool as a Bazel execution platform. The platform's exec_properties describe the properties of the pool. A constraint value then lets targets select that execution platform with exec_compatible_with:

constraint_setting(name = "worker_pool")
 
constraint_value(
    name = "compile_pool_constraint",
    constraint_setting = ":worker_pool",
)
 
constraint_value(
    name = "high_memory_pool_constraint",
    constraint_setting = ":worker_pool",
)
 
platform(
    name = "compile_pool",
    constraint_values = [":compile_pool_constraint"],
    exec_properties = {
        "namespace_pool": "compile",
        "namespace_pool_machine_type": "linux/amd64:16x32",
        "namespace_pool_slots": "4",
    },
)
 
platform(
    name = "high_memory_pool",
    constraint_values = [":high_memory_pool_constraint"],
    exec_properties = {
        "namespace_pool": "high-memory",
        "namespace_pool_machine_type": "linux/amd64:16x64",
        "namespace_pool_slots": "2",
    },
)
 
genrule(
    name = "compile_probe",
    outs = ["compile_probe.txt"],
    cmd = "printf compile > $@",
 
    exec_compatible_with = [":compile_pool_constraint"],
)
 
genrule(
    name = "high_memory_probe",
    outs = ["high_memory_probe.txt"],
    cmd = "printf high-memory > $@",
 
    exec_compatible_with = [":high_memory_pool_constraint"],
)

Register the pool platforms when invoking Bazel:

$ bazel --bazelrc=~/.namespace.bazelrc build //... \
  --extra_execution_platforms=//:compile_pool,//:high_memory_pool

You can add --extra_execution_platforms to your project bazelrc instead of passing it on every invocation.

The machine type uses the format <os>/<arch>:<cpu>x<memory>. For example, linux/amd64:16x64 selects a Linux AMD64 worker with 16 vCPUs and 64 GB of memory. See Machine Shapes for supported CPU and memory configurations.

The following execution properties define a pool:

Every execution platform that uses the same pool name must provide the same pool definition. The scheduler rejects an action if its machine type, slot count, or worker image conflicts with the pool's existing definition.

Custom worker base images

By default, actions run using one of Namespace's managed worker images. If your actions need extra tools, libraries, or a specific base environment, you can run them inside your own container image instead.

Custom images are selected per action through the standard Bazel container-image execution property.

A few requirements apply:

• The image reference must be pinned to an immutable digest (repo@sha256:...). Tags are rejected.
• The image must be optimized before it can be used. Namespace converts the image into a fast-booting disk variant.
• The worker image must be hosted in the Namespace container registry nscr.io.
• Custom images are supported on Linux workers only.

Build, push, and optimize a custom image

$ nsc docker login
$ nsc build . -t nscr.io/<tenant>/my-worker:latest --platform linux/amd64 --push

$ nsc base-image upload <source-image> my-worker:latest

$ docker inspect --format '{{index .RepoDigests 0}}' nscr.io/<tenant>/my-worker:latest
nscr.io/<tenant>/my-worker@sha256:<digest>

$
nsc base-image optimize --image_ref nscr.io/<tenant>/my-worker@sha256:<digest>

jobs:
  rbe:
    runs-on:
      - namespace-profile-foobar;permissions.additional_grant=baseimage:*:*

jobs:
  rbe:
    runs-on:
      - nscloud-ubuntu-22.04-amd64-4x8-with-features
      - namespace-features:permissions.additional_grant=baseimage:*:*

Use the custom image

Set the container-image execution property on a specific target or platform so only those actions run with the custom worker image:

exec_properties = {
    "container-image": "docker://<repo>@sha256:<digest>",
}

The docker:// scheme prefix is optional. To apply an image through a custom worker pool instead, set namespace_pool_worker_image as described in Custom worker pools.

Run preparation scripts at worker startup

A custom worker image can ship preparation scripts that run once before the worker starts handling any actions. These hooks handle setup that must run on every worker boot, such as mounting an external /nix store or seeding caches.

Place executable files in /etc/namespace/prepare.d inside your image. On startup, the worker runs every executable file in that directory in lexical order (e.g. 10-mount.sh before 20-warmup.sh). Non-executable files and subdirectories are skipped.

Observability

Namespace records every invocation that runs through remote execution. Inspect them from the Bazel invocations page, or from the CLI with nsc bazel invocation list and nsc bazel invocation report.

Reports carry worker assignment and timing, queue times, and hydration times alongside the usual action and target data, which makes them the place to look when a build fails for a reason Bazel cannot see.

See Bazel observability for the full workflow, including how to analyze a report with an agent.

Future reports will also include operation joins, which indicate whether an invocation joins action invocations from other invocations, and hot or warm cache information.

Current caveats

Remote Execution has the following current limitations:

• Action isolation within macOS workers is not available yet.

If any of these caveats block your use case, contact support@namespace.so so we can help find an appropriate solution.