Ei kuvausta

Nikolay Sivko 4c54df0c81 python: search for `pthread_cond_timedwait` in both libc.so and libpthread.so 1 vuosi sitten
.github 05c838332c GH actions: upload agent binaries to GH releases 2 vuotta sitten
cgroup b8211bf0ac fix container RSS memory calculation with cgroup v2 stats. 1 vuosi sitten
common 16f8e53467 enable tracking of all connections by default (0.0.0.0/0) 2 vuotta sitten
containers 9009c5c96c add `container_python_thread_lock_wait_time_seconds` metric 1 vuosi sitten
ebpftracer 4c54df0c81 python: search for `pthread_cond_timedwait` in both libc.so and libpthread.so 1 vuosi sitten
flags 16f8e53467 enable tracking of all connections by default (0.0.0.0/0) 2 vuotta sitten
logs f2b9d57659 metrics push mode using the Prometheus Remote Write API 2 vuotta sitten
manifests 92ef04b486 squashing commits before publishing 4 vuotta sitten
node c7c218978b goimports: imports reformatting 2 vuotta sitten
pinger f98e1c18c5 pinger: read RX timestamp only upon receiving a valid ICMP Echo reply 2 vuotta sitten
proc 0607a535b4 init: don't ignore errors 2 vuotta sitten
profiling 44f6e37e99 profiling: set `SampleType` name 2 vuotta sitten
prom dff2224ce1 metrics: set `info` log level 2 vuotta sitten
tracing 341cef08b8 tracing: retrieve `ServiceName` from `ContainerId` 2 vuotta sitten
.dockerignore 12d24bf447 capturing Postgres queries at the eBPF level 3 vuotta sitten
.gitignore 92ef04b486 squashing commits before publishing 4 vuotta sitten
CONTRIBUTING.md 1099fdd351 contributing guide 2 vuotta sitten
Dockerfile 48e3d5f44e continuous integration 2 vuotta sitten
LICENSE 92ef04b486 squashing commits before publishing 4 vuotta sitten
Makefile 1099fdd351 contributing guide 2 vuotta sitten
README.md 1099fdd351 contributing guide 2 vuotta sitten
go.mod 41714cd50b Add DNS protocol tracing 2 vuotta sitten
go.sum 41714cd50b Add DNS protocol tracing 2 vuotta sitten
install.sh f55e13dca3 Add script to install agent as a Systemd service 2 vuotta sitten
main.go 6acd550f3b metrics: add `system_uuid` label 2 vuotta sitten

README.md

Coroot-node-agent

Go Report Card License

The agent gathers metrics related to a node and the containers running on it, and it exposes them in the Prometheus format.

It uses eBPF to track container related events such as TCP connects, so the minimum supported Linux kernel version is 4.16.

Features

TCP connection tracing

To provide visibility into the relationships between services, the agent traces containers TCP events, such as connect() and listen().

Exported metrics are useful for:

  • Obtaining an actual map of inter-service communications. It doesn't require integration of distributed tracing frameworks into your code.
  • Detecting connections errors from one service to another.
  • Measuring network latency between containers, nodes and availability zones.

Related blog posts:

Log management is usually quite expensive. In most cases, you do not need to analyze each event individually. It is enough to extract recurring patterns and the number of the related events.

This approach drastically reduces the amount of data required for express log analysis.

The agent discovers container logs and parses them right on the node.

At the moment the following sources are supported:

  • Direct logging to files in /var/log/
  • Journald
  • Dockerd (JSON file driver)
  • Containerd (CRI logs)

To learn more about automated log clustering, check out the blog post "Mining metrics from unstructured logs".

Delay accounting

Delay accounting allows engineers to accurately identify situations where a container is experiencing a lack of CPU time or waiting for I/O.

The agent gathers per-process counters through Netlink and aggregates them into per-container metrics:

Related blog posts:

Out-of-memory events tracing

The container_oom_kills_total metric shows that a container has been terminated by the OOM killer.

Instance meta information

If a node is a cloud instance, the agent identifies a cloud provider and collects additional information using the related metadata services.

Supported cloud providers: AWS, GCP, Azure, Hetzner

Collected info:

  • AccountID
  • InstanceID
  • Instance/machine type
  • Region
  • AvailabilityZone
  • AvailabilityZoneId (AWS only)
  • LifeCycle: on-demand/spot (AWS and GCP only)
  • Private & Public IP addresses

Related blog posts:

Installation

The documentation is available at coroot.com/docs/metric-exporters/node-agent.

Metrics

The collected metrics are described here.

Coroot

The best way to turn metrics to answers about app issues is to use Coroot - a zero-instrumentation observability tool for microservice architectures.

A live demo of Coroot is available at community-demo.coroot.com

Contributing

To start contributing, check out our Contributing Guide.

License

Coroot-node-agent is licensed under the Apache License, Version 2.0.

The BPF code is licensed under the General Public License, Version 2.0.