Bottlerocket OS

Welcome to Bottlerocket!

Bottlerocket is a free and open-source Linux-based operating system meant for hosting containers.

To learn more about Bottlerocket, visit the official Bottlerocket website and documentation. Otherwise, if you’re ready to jump right in, read one of our setup guides for running Bottlerocket in Amazon EKS, Amazon ECS, or VMware. If you're interested in running Bottlerocket on bare metal servers, please refer to the provisioning guide to get started.

Bottlerocket focuses on security and maintainability, providing a reliable, consistent, and safe platform for container-based workloads. This is a reflection of what we've learned building operating systems and services at Amazon. You can read more about what drives us in our charter.

The base operating system has just what you need to run containers reliably, and is built with standard open-source components. Bottlerocket-specific additions focus on reliable updates and on the API. Instead of making configuration changes manually, you can change settings with an API call, and these changes are automatically migrated through updates.

Some notable features include:

API access for configuring your system, with secure out-of-band access methods when you need them.
Updates based on partition flips, for fast and reliable system updates.
Modeled configuration that's automatically migrated through updates.
Security as a top priority.

Participate in the Community

There are many ways to take part in the Bottlerocket community:

Join us on Meetup to hear about the latest Bottlerocket (virtual/in-person) events and community meetings. Community meetings are typically every other week.

Details can be found under the Events section on Meetup, and you will receive email notifications if you become a member of the Meetup group. (It's free to join!)
Start or join a discussion if you have questions about Bottlerocket.
If you're interested in contributing, thank you! Please see our contributor's guide.

Contact us

If you find a security issue, please contact our security team rather than opening an issue.

We use GitHub issues to track other bug reports and feature requests. You can look at existing issues to see whether your concern is already known.

If not, you can select from a few templates and get some guidance on the type of information that would be most helpful. Contact us with a new issue here.

We don't have other communication channels set up quite yet, but don't worry about making an issue or a discussion thread! You can let us know about things that seem difficult, or even ways you might like to help.

Variants

To start, we're focusing on the use of Bottlerocket as a host OS in AWS EKS Kubernetes clusters and Amazon ECS clusters. We’re excited to get early feedback and to continue working on more use cases!

Bottlerocket is architected such that different cloud environments and container orchestrators can be supported in the future. A build of Bottlerocket that supports different features or integration characteristics is known as a 'variant'. The artifacts of a build will include the architecture and variant name. For example, an x86_64 build of the aws-k8s-1.24 variant will produce an image named bottlerocket-aws-k8s-1.24-x86_64-<version>-<commit>.img.

The following variants support EKS, as described above:

aws-k8s-1.23
aws-k8s-1.24
aws-k8s-1.25
aws-k8s-1.26
aws-k8s-1.27
aws-k8s-1.28
aws-k8s-1.29
aws-k8s-1.30
aws-k8s-1.23-nvidia
aws-k8s-1.24-nvidia
aws-k8s-1.25-nvidia
aws-k8s-1.26-nvidia
aws-k8s-1.27-nvidia
aws-k8s-1.28-nvidia
aws-k8s-1.29-nvidia
aws-k8s-1.30-nvidia

The following variants support ECS:

aws-ecs-1
aws-ecs-1-nvidia
aws-ecs-2
aws-ecs-2-nvidia

We also have variants that are designed to be Kubernetes worker nodes in VMware:

vmware-k8s-1.28
vmware-k8s-1.29
vmware-k8s-1.30

The following variants are designed to be Kubernetes worker nodes on bare metal:

metal-k8s-1.28
metal-k8s-1.29

The following variants are no longer supported:

All Kubernetes variants using Kubernetes 1.22 and earlier
Bare metal and VMware variants using Kubernetes 1.27 and earlier

We recommend users replace nodes running these variants with the latest variant compatible with their cluster.

Architectures

Our supported architectures include x86_64 and aarch64 (written as arm64 in some contexts).

Setup

:walking: :running:

Bottlerocket is best used with a container orchestrator. To get started with Kubernetes in Amazon EKS, please see QUICKSTART-EKS. To get started with Kubernetes in VMware, please see QUICKSTART-VMWARE. To get started with Amazon ECS, please see QUICKSTART-ECS. These guides describe:

how to set up a cluster with the orchestrator, so your Bottlerocket instance can run containers
how to launch a Bottlerocket instance in EC2 or VMware

To see how to provision Bottlerocket on bare metal, see PROVISIONING-METAL.

To build your own Bottlerocket images, please see BUILDING. It describes:

how to build an image
how to register an EC2 AMI from an image

To publish your built Bottlerocket images, please see PUBLISHING. It describes:

how to make TUF repos including your image
how to copy your AMI across regions
how to mark your AMIs public or grant access to specific accounts
how to make your AMIs discoverable using SSM parameters

Exploration

To improve security, there's no SSH server in a Bottlerocket image, and not even a shell.

Don't panic!

There are a couple out-of-band access methods you can use to explore Bottlerocket like you would a typical Linux system. Either option will give you a shell within Bottlerocket. From there, you can change settings, manually update Bottlerocket, debug problems, and generally explore.

Note: These methods require that your instance has permission to access the ECR repository where these containers live; the appropriate policy to add to your instance's IAM role is AmazonEC2ContainerRegistryReadOnly.

Control container

Bottlerocket has a "control" container, enabled by default, that runs outside of the orchestrator in a separate instance of containerd. This container runs the AWS SSM agent that lets you run commands, or start shell sessions, on Bottlerocket instances in EC2. (You can easily replace this control container with your own just by changing the URI; see Settings.)

In AWS, you need to give your instance the SSM role for this to work; see the setup guide. Outside of AWS, you can use AWS Systems Manager for hybrid environments. There's more detail about hybrid environments in the control container documentation.

Once the instance is started, you can start a session:

Go to AWS SSM's Session Manager
Select "Start session" and choose your Bottlerocket instance
Select "Start session" again to get a shell

If you prefer a command-line tool, you can start a session with a recent AWS CLI and the session-manager-plugin. Then you'd be able to start a session using only your instance ID, like this:

aws ssm start-session --target INSTANCE_ID --region REGION_CODE

With the default control container, you can make API calls to configure and manage your Bottlerocket host. To do even more, read the next section about the admin container. You can access the admin container from the control container like this:

enter-admin-container

Admin container

Bottlerocket has an administrative container, disabled by default, that runs outside of the orchestrator in a separate instance of containerd. This container has an SSH server that lets you log in as ec2-user using your EC2-registered SSH key. Outside of AWS, you can pass in your own SSH keys. (You can easily replace this admin container with your own just by changing the URI; see Settings.

To enable the container, you can change the setting in user data when starting Bottlerocket, for example EC2 instance user data:

[settings.host-containers.admin]
enabled = true

If Bottlerocket is already running, you can enable the admin container from the default control container like this:

enable-admin-container

Or you can start an interactive session immediately like this:

enter-admin-container

If you're using a custom control container, or want to make the API calls directly, you can enable the admin container like this instead:

apiclient set host-containers.admin.enabled=true

Once you've enabled the admin container, you can either access it through SSH or execute commands from the control container like this:

apiclient exec admin bash

Once you're in the admin container, you can run sheltie to get a full root shell in the Bottlerocket host. Be careful; while you can inspect and change even more as root, Bottlerocket's filesystem and dm-verity setup will prevent most changes from persisting over a restart - see Security.

Updates

Rather than a package manager that updates individual pieces of software, Bottlerocket downloads a full filesystem image and reboots into it. It can automatically roll back if boot failures occur, and workload failures can trigger manual rollbacks.

The update process uses images secured by TUF. For more details, see the update system documentation.

Update methods

There are several ways of updating your Bottlerocket hosts. We provide tools for automatically updating hosts, as well as an API for direct control of updates.

Automated updates

For EKS variants of Bottlerocket, we recommend using the Bottlerocket update operator for automated updates.

For the ECS variant of Bottlerocket, we recommend using the Bottlerocket ECS updater for automated updates.

Update API

The Bottlerocket API includes methods for checking and starting system updates. You can read more about the update APIs in our update system documentation.

apiclient knows how to handle those update APIs for you, and you can run it from the control or admin containers.

To see what updates are available:

apiclient update check

If an update is available, it will show up in the chosen_update field. The available_updates field will show the full list of available versions, including older versions, because Bottlerocket supports safely rolling back.

To apply the latest update:

apiclient update apply

The next time you reboot, you'll start up in the new version, and system configuration will be automatically migrated. To reboot right away:

apiclient reboot

If you're confident about updating, the apiclient update apply command has --check and --reboot flags to combine the above actions, so you can accomplish all of the above steps like this:

apiclient update apply --check --reboot

See the apiclient documentation for more details.

Update rollback

The system will automatically roll back if it's unable to boot. If the update is not functional for a given container workload, you can do a manual rollback:

signpost rollback-to-inactive
reboot

This doesn't require any external communication, so it's quicker than apiclient, and it's made to be as reliable as possible.

Settings

Here we'll describe the settings you can configure on your Bottlerocket instance, and how to do it.

(API endpoints are defined in our OpenAPI spec if you want more detail.)

Interacting with settings

Using the API client

You can see the current settings with an API request:

apiclient get settings

This will return all of the current settings in JSON format. For example, here's an abbreviated response:

{"motd": "...", {"kubernetes": {}}}

You can change settings like this:

apiclient set motd="hi there" kubernetes.node-labels.environment=test

You can also use a JSON input mode to help change many related settings at once, and a "raw" mode if you want more control over how the settings are committed and applied to the system. See the apiclient README for details.

Using user data

If you know what settings you want to change when you start your Bottlerocket instance, you can send them in the user data.

In user data, we structure the settings in TOML form to make things a bit simpler. Here's the user data to change the message of the day setting, as we did in the last section:

[settings]
motd = "my own value!"

If your user data is over the size limit of the platform (e.g. 16KiB for EC2) you can compress the contents with gzip. (With aws-cli, you can use --user-data fileb:///path/to/gz-file to pass binary data.)

Description of settings

Here we'll describe each setting you can change.

Note: You can see the default values (for any settings that are not generated at runtime) by looking in the defaults.d directory for a variant, for example aws-ecs-2.

When you're sending settings to the API, or receiving settings from the API, they're in a structured JSON format. This allows modification of any number of keys at once. It also lets us ensure that they fit the definition of the Bottlerocket data model - requests with invalid settings won't even parse correctly, helping ensure safety.

Here, however, we'll use the shortcut "dotted key" syntax for referring to keys. This is used in some API endpoints with less-structured requests or responses. It's also more compact for our needs here.

In this format, "settings.kubernetes.cluster-name" refers to the same key as in the JSON {"settings": {"kubernetes": {"cluster-name": "value"}}}.

NOTE: bottlerocket.dev now contains a complete, versioned setting reference. This documents retains the headings below for existing link and bookmark compatability. Please update your bookmarks and check out bottlerocket.dev for future updates to the setting reference.