This repository implements a whole-system, cross-language profiler for Linux via eBPF.
.eh_frame data as described in US11604718B1)inline frames, which provide insights into compiler optimizations and offer a higher precision of function call chains.The minimum required Linux kernel version has increased with certain commits. Specifically:
The project maintains its minimum supported kernel version in line with the lowest kernel version currently provided by actively maintained major Linux distributions, which include Debian stable, Red Hat Enterprise Linux, Ubuntu LTS, Amazon Linux and SUSE Linux. The minimum requirement may be increased when all such distributions no longer ship a specific kernel version. This approach enables the codebase to utilize newer eBPF features and avoids the need to maintain compatibility shims for obsolete kernels.
It should be noted that certain distributions incorporate eBPF features from newer kernels into their supported versions. When this occurs, the distribution's stated kernel version does not accurately reflect its true eBPF capabilities and will not prevent us from increasing the minimum supported version. On such kernels, the no-kernel-version-check configuration option can be used to bypass the checks and allow the profiler to execute.
We have integrated the profiler into the OTel Collector as a receiver, and this is the supported configuration going forward.
To aid with development, testing and debugging, we also offer a standalone profiling agent binary named ebpf-profiler, and a local build of an OTel Collector profiling receiver binary (otelcol-ebpf-profiler). These binaries are not supported in any way, can be dropped in the future and should not be deployed in production.
The agent can be built with the provided make targets. Docker is required for containerized builds, and both amd64 and arm64 architectures are supported.
For Linux, the following steps apply:
make agent
make agent TARGET_ARCH=arm64
The resulting binary will be named ebpf-profiler in the current directory.
Since the profiler is Linux-only, macOS and Windows users need to set up a Linux VM to build and run the agent. Ensure the appropriate architecture is specified if using cross-compilation. Use the same make targets as above after the Linux environment is configured in the VM.
You can build the agent without Docker by directly installing the dependencies listed in the Dockerfile. Once dependencies are set up, simply run:
make
This will build the profiler natively on your machine.
otelcol-ebpf-profiler locally (Without Docker)You can build the local otelcol-ebpf-profiler binary by running:
make otelcol-ebpf-profiler
or to cross-compile for a different architecture (e.g. arm64):
make otelcol-ebpf-profiler TARGET_ARCH=arm64
See local.example.yaml for an example configuration.
You can start the agent with the following command:
sudo ./ebpf-profiler -collection-agent=127.0.0.1:11000 -disable-tls
To start the OTel Collector profiling receiver, run:
sudo ./otelcol-ebpf-profiler --feature-gates=+service.profilesSupport --config cmd/otelcol-ebpf-profiler/local.example.yaml
The agent comes with a functional but work-in-progress / evolving implementation of the recently released Alpha OTel Profiles signal.
The agent loads the eBPF program and its maps, starts unwinding and reports captured traces to the backend.
As the OTel Profiles signal is still in development, mature production-ready backends have yet to emerge. The following open source projects can be used as backends:
We welcome contributions of all kinds: bug reports, feature requests, documentation improvements and code. Everyone is welcome to open issues and send pull requests.
Before contributing, please read the contributing guidelines. They cover the development prerequisites, how to build and test your changes, the pull request workflow, and how PRs get reviewed and merged.
To understand how this project works and learn more about profiling, check out Profiling internals.
This project is developed by the OpenTelemetry Profiling SIG, which meets regularly. See the OpenTelemetry community repository for the meeting schedule, and join us in the #otel-profiles channel on the CNCF Slack for discussions and questions.
For more information about the maintainer role, see the community repository.
For more information about the approver role, see the community repository.
For more information about the emeritus role, see the community repository.
See the community membership document in OpenTelemetry community repo.
This project is licensed under the Apache License 2.0 (Apache-2.0). Apache License 2.0
The eBPF source code is licensed under the GPL 2.0 license. GPL 2.0
To display a summary of the dependencies' licenses:
make legal