Confusion with InfluxDB/Grafana Data

Hello! I am running k6 on multiple servers. Each running instance is configured to send the metrics to a central InfluxDB instance, which is read/graphed by Grafana. Everything has been working perfectly so far! However, I am having trouble configuring Grafanna to show correct data.

I understand this is not directly a k6 question, however I’m hoping it’s still appropriate to post here.

The issue: I have k6 running on 16 separate servers. The test script is configured to ramp up to a max of 500 VUs.

The right graph is correct: 500 VUs per server. However the left graph is incorrect. There should be 16 * 500 = 8,000 not 40,000 as the graph displays.

Here’s the query I’m using:

SELECT sum(“value”) FROM “vus” WHERE timeFilter GROUP BY time(__interval) fill(null)

Am I using the vus.sum value incorrectly? This also makes me worry that my other graphs (http_reqs.sum, data_sent.sum, and data_received.sum) are also incorrect.

Thank you!
Steve

My InfluxDB knowledge is very basic, so I might be mistaken, but I think your issue might be caused by GROUP BY time(__interval). Each k6 instance will emit the vus metric every second, so if your __interval is 5 seconds, that could be the reason you’re seeing 5 times as many VUs as you should.

I’d suggest tagging the metrics out of each instance with the --tag option, like this:

k6 run --tag instance_id=1 script.js
k6 run --tag instance_id=2 script.js
# ...
k6 run --tag instance_id=16 script.js

And then using some InfluxDB query magic that allows you to differentiate between the instances when you need to. For most things, you probably won’t need to care, since they reflect actual events like an HTTP request or an iteration finishing. But vus and vus_max are emitted periodically by each instance…

Hi @ned,

The interval in the query not matching k6’s report interval makes sense! Does k6 report everything once a second? Is that value configurable?

Thanks for pointing me in the right direction! I will also investigate using the --tag as well.

Thanks,
Steve

Just to close out this thread, I was able to solve my issue by using this query instead:

SELECT sum("value") FROM "vus" WHERE $timeFilter GROUP BY time(1s) fill(null)

This overrides Grafana’s $interval which is based on the width of the graph and the selected time window.

No, most metrics are emitted with the time they are generated, e.g. http_req_duration is emitted as soon as the HTTP request is over. vus and vus_max are the exceptions, being emitted every second, and unfortunately that’s not configurable for now.