Jaeger - a Distributed Tracing System

💥💥💥 Jaeger v2 is coming! Read the blog post and try it out.

graph TD
    SDK["OpenTelemetry SDK"] --> |HTTP or gRPC| COLLECTOR
    COLLECTOR["Jaeger Collector"] --> STORE[Storage]
    COLLECTOR --> |gRPC| PLUGIN[Storage Plugin]
    COLLECTOR --> |gRPC/sampling| SDK
    PLUGIN --> STORE
    QUERY[Jaeger Query Service] --> STORE
    QUERY --> |gRPC| PLUGIN
    UI[Jaeger UI] --> |HTTP| QUERY
    subgraph Application Host
        subgraph User Application
            SDK
        end
    end

Jaeger, inspired by Dapper and OpenZipkin, is a distributed tracing platform created by Uber Technologies and donated to Cloud Native Computing Foundation. It can be used for monitoring microservices-based distributed systems:

Distributed context propagation
Distributed transaction monitoring
Root cause analysis
Service dependency analysis
Performance / latency optimization

Get Involved

Jaeger is an open source project with open governance. We welcome contributions from the community, and we would love your help to improve and extend the project. Here are some ideas for how to get involved. Many of them do not even require any coding.

Features

High Scalability

Jaeger backend is designed to have no single points of failure and to scale with the business needs. For example, any given Jaeger installation at Uber is typically processing several billions of spans per day.

Relationship with OpenTelemetry

The Jaeger and OpenTelemetry projects have different goals. OpenTelemetry aims to provide APIs and SDKs in multiple languages to allow applications to export various telemetry data out of the process, to any number of metrics and tracing backends. The Jaeger project is primarily the tracing backend that receives tracing telemetry data and provides processing, aggregation, data mining, and visualizations of that data. For more information please refer to a blog post Jaeger and OpenTelemetry.

Jaeger was originally designed to support the OpenTracing standard. The terminology is still used in Jaeger UI, but the concepts have direct mapping to the OpenTelemetry data model of traces.

Capability	OpenTracing concept	OpenTelemetry concept
Represent traces as directed acyclic graphs (not just trees)	span references	span links
Strongly typed span attributes	span tags	span attributes
Strongly typed events/logs	span logs	span events

Jaeger project recommends OpenTelemetry SDKs for instrumentation, instead of now-deprecated Jaeger SDKs.

Multiple storage backends

Jaeger can be used with a growing a number of storage backends:

It natively supports two popular open source NoSQL databases as trace storage backends: Cassandra and Elasticsearch.
It integrates via a gRPC API with other well known databases that have been certified to be Jaeger compliant: TimescaleDB via Promscale, ClickHouse.
There is embedded database support using Badger and simple in-memory storage for testing setups.
ScyllaDB can be used as a drop-in replacement for Cassandra since it uses the same data model and query language.
There are ongoing community experiments using other databases, such as InfluxDB, Amazon DynamoDB, YugabyteDB(YCQL).

Modern Web UI

Jaeger Web UI is implemented in Javascript using popular open source frameworks like React. Several performance improvements have been released in v1.0 to allow the UI to efficiently deal with large volumes of data and to display traces with tens of thousands of spans (e.g. we tried a trace with 80,000 spans).

Cloud Native Deployment

Jaeger backend is distributed as a collection of Docker images. The binaries support various configuration methods, including command line options, environment variables, and configuration files in multiple formats (yaml, toml, etc.).

The recommended way to deploy Jaeger in a production Kubernetes cluster is via the Jaeger Operator.

The Jaeger Operator provides a CLI to generate Kubernetes manifests from the Jaeger CR. This can be considered as an alternative source over plain Kubernetes manifest files.

The Jaeger ecosystem also provides a Helm chart as an alternative way to deploy Jaeger.

Observability

All Jaeger backend components expose Prometheus metrics by default (other metrics backends are also supported). Logs are written to standard out using the structured logging library zap.

Security

Third-party security audits of Jaeger are available in https://github.com/jaegertracing/security-audits. Please see Issue #1718 for the summary of available security mechanisms in Jaeger.

Backwards compatibility with Zipkin

Although we recommend instrumenting applications with OpenTelemetry, if your organization has already invested in the instrumentation using Zipkin libraries, you do not have to rewrite all that code. Jaeger provides backwards compatibility with Zipkin by accepting spans in Zipkin formats (Thrift or JSON v1/v2) over HTTP. Switching from Zipkin backend is just a matter of routing the traffic from Zipkin libraries to the Jaeger backend.

Version Compatibility Guarantees

Occasionally, CLI flags can be deprecated due to, for example, usability improvements or new functionality. In such situations, developers introducing the deprecation are required to follow these guidelines.

In short, for a deprecated CLI flag, you should expect to see the following message in the --help documentation:

(deprecated, will be removed after yyyy-mm-dd or in release vX.Y.Z, whichever is later)

A grace period of at least 3 months or two minor version bumps (whichever is later) from the first release containing the deprecation notice will be provided before the deprecated CLI flag can be deleted.

For example, consider a scenario where v1.28.0 is released on 01-Jun-2021 containing a deprecation notice for a CLI flag. This flag will remain in a deprecated state until the later of 01-Sep-2021 or v1.30.0 where it can be removed on or after either of those events. It may remain deprecated for longer than the aforementioned grace period.

Go Version Compatibility Guarantees

The Jaeger project attempts to track the currently supported versions of Go, as defined by the Go team. Removing support for an unsupported Go version is not considered a breaking change.

Starting with the release of Go 1.21, support for Go versions will be updated as follows:

Soon after the release of a new Go minor version N, updates will be made to the build and tests steps to accommodate the latest Go minor version.
Soon after the release of a new Go minor version N, support for Go version N-2 will be removed and version N-1 will become the minimum required version.

Related Repositories

Documentation

Published: https://www.jaegertracing.io/docs/
Source: https://github.com/jaegertracing/documentation

Instrumentation Libraries

Jaeger project recommends OpenTelemetry SDKs for instrumentation, instead of Jaeger's native SDKs that are now deprecated.

Deployment

Jaeger Operator for Kubernetes

Components

Building From Source

See CONTRIBUTING.

Contributing

See CONTRIBUTING.

Thanks to all the people who already contributed!

Maintainers

Rules for becoming a maintainer are defined in the GOVERNANCE document. Below are the official maintainers of the Jaeger project. Please use @jaegertracing/jaeger-maintainers to tag them on issues / PRs.

Some repositories under jaegertracing org have additional maintainers.

Emeritus Maintainers

We are grateful to our former maintainers for their contributions to the Jaeger project.

Project Status Meetings

The Jaeger maintainers and contributors meet regularly on a video call. Everyone is welcome to join, including end users. For meeting details, see https://www.jaegertracing.io/get-in-touch/.

Roadmap

See https://www.jaegertracing.io/docs/roadmap/

Get in Touch

Have questions, suggestions, bug reports? Reach the project community via these channels:

Slack chat room #jaeger (need to join CNCF Slack for the first time)
jaeger-tracing mail group
GitHub issues and discussions

Adopters

Jaeger as a product consists of multiple components. We want to support different types of users, whether they are only using our instrumentation libraries or full end to end Jaeger installation, whether it runs in production or you use it to troubleshoot issues in development.

Please see ADOPTERS.md for some of the organizations using Jaeger today. If you would like to add your organization to the list, please comment on our survey issue.