My k6 load test runs on multiple Kubernetes pods (to simulate heavy load). It works fine with K6_OUT=influxdb, but when “datadog” out is used, there are issues with aggregating data.
Specifically, I can see that counters (vus, vus_max, http_reqs etc…) are not aggregated from multiple sources (k6 worker pods) and only single pod metrics are displayed. Datadog dashboard is set to “sum” counters, but it looks like the problem is in datadog-agent/k6 integration.
Current k8s design is following:
- single datadog-agent pod that exposed as a service
- multiple k6 workers (e.g. 30 instances) that have K6_OUT=datadog, K6_DATADOG_ADDR=[cluster-dns-name]]
Any ideas what’s wrong here?
I suspect that the problem can be fixed if each k6 worker will forward stats to individual datadog-agent (per node), but that increases complexity and Datadog will charge more (since pricing it based on number of agents).