I see that it is possible to make some custom thresholds in k6. Is it possible to make such thresholds that will abort the test if the rps of the script stops increasing? For example, vus could not generate more rps at some stage of the test, and this is a certain limit, and therefore an abortion of the test?
Or p95 response time of the group out of limit within, some time, 10 minutes for example. Something like that - p95 response time of the group has exceeded and we fix this moment, and after 10 minutes, if p95 has not fallen below the specified value, then abort the test.
It is possible?
Thanks for your advices.
I discussed this with @mstoykov and we don’t think there is anything k6 can do in this case.
Thresholds are for the whole execution only. If you have stages that are tagging the requests and those were 10 minutes long, you could check that for a given stage the rps value, but not compare them.
This can be calculated in something like Grafana if the metric is sent to a data storage like Prometheus, InfluxDb, etc. However, you’d still need to stop the test based on the values you see in Grafana.
k6 cloud has a performance insight to tell you that the “VUs were increased but the RPS did not increase or it did not increase “linearly” with the number of VUs”.
For the need to stop the test if the rate is not reachable at some point, k6 outputs a warning in the logs:
WARN Insufficient VUs, reached 100 active VUs and cannot initialize more executor=constant-arrival-rate scenario=test
Thanks for raising this questions.