Repolode
Submit
facebookincubator

facebookincubator/dynolog

A lightweight telemetry daemon that exports Linux, CPU, GPU and PyTorch training metrics and triggers on-demand traces.

View on GitHub
Stars
388
Forks
94
Open issues
33
Last commit
20 hours ago

What is it?

Dynolog is a monitoring daemon for heterogeneous CPU-GPU systems, built for both always-on performance monitoring and deep-dive profiling. Profiling modes are activated by a remote procedure call to the daemon, and a single `dyno` command line tool talks to it on the local or a remote host.

It collects kernel metrics covering CPU, network and IO usage, hardware PMU counters from Linux perf_event on Intel and AMD CPUs, and NVIDIA GPU metrics through DCGM. Metrics can be written to standard output, to OTLP endpoints, or to Meta's internal ODS and Scuba systems through a pluggable Logger class.

For PyTorch workloads, dynolog integrates with the PyTorch Profiler (v1.13.0 and later) so that traces can be triggered on demand across many GPUs, including coordinated tracing in Slurm-managed distributed training jobs. It targets Linux only, with RPM and Debian packages as well as source builds.

Key features

  • 01Always-on kernel, CPU and PMU metric collection on Linux.
  • 02On-demand PyTorch Profiler traces triggered remotely via the dyno CLI.
  • 03NVIDIA GPU monitoring through DCGM 2.x and 3.x.
  • 04OTLP metrics export to OpenTelemetry-compatible backends.
  • 05Pluggable logger for ODS, Scuba, OTLP or stdout.

Use cases

  • Tracing slow iterations in large distributed PyTorch training jobs across many GPUs.
  • Monitoring CPU, network and IO health on Linux servers in production.
  • Collecting GPU metrics from NVIDIA data center hosts into an existing observability stack.

Quick start

  1. 1

    Download the release package for your distro from the releases page

  2. 2

    Install the Debian package on Ubuntu or Debian

    sudo dpkg -i dynolog_0.2.1-0-amd64.deb
  3. 3

    Start the dynolog service with systemd

    sudo systemctl start dynolog
  4. 4

    Check that the daemon is running with the dyno CLI

    dyno status

Things to know

  • Linux only; non-Linux platforms are not supported.
  • GPU monitoring works only with NVIDIA GPUs and requires separately installing DCGM.
  • GPU trace collection requires PyTorch v1.13.0 or a compatible version.

In collections

Run and train AI on GPUsOpenTelemetry & observability platforms

Topics

#performance-monitoring#profiling#gpu#pytorch#observability#linux#telemetry

FAQ

What is dynolog?
A lightweight telemetry daemon that exports Linux, CPU, GPU and PyTorch training metrics and triggers on-demand traces. Dynolog is a monitoring daemon for heterogeneous CPU-GPU systems, built for both always-on performance monitoring and deep-dive profiling. Profiling modes are activated by a remote procedure call to the daemon, and a single `dyno` command line tool talks to it on the local or a remote host.
Is dynolog free to use? What license does it have?
dynolog is open source under the MIT license. Check the license terms before using it in commercial products.
Is dynolog still maintained?
Actively maintained. The last commit to dynolog was on Nov 7, 2026.
What can I use dynolog for?
Tracing slow iterations in large distributed PyTorch training jobs across many GPUs. Monitoring CPU, network and IO health on Linux servers in production. Collecting GPU metrics from NVIDIA data center hosts into an existing observability stack.
How do I install dynolog?
Download the release package for your distro from the releases page Install the Debian package on Ubuntu or Debian: sudo dpkg -i dynolog_0.2.1-0-amd64.deb Start the dynolog service with systemd: sudo systemctl start dynolog Check that the daemon is running with the dyno CLI: dyno status
Where can I get dynolog?
The source code and installation instructions are on GitHub: https://github.com/facebookincubator/dynolog

Summary generated by AI from the project's README. GitHub stats refresh daily.

Related projects