Scenario includes high rate of UDP calls, is k6 worth considering?

I’m planning a load test on a system which has the VUs exchange UDP packets based on a scenario (with http calls as well). I started looking into k6 and found it very appealing, and I though I could use the xk6 extensions to add some UDP capabilities here.

Then I read the page about large tests, and I’m wondering if k6 is worth considering anymore as it does not support clustering. Each of my VUs must be able to send or receive hundreds of small UDP packets per second, and I must be able to simulate millions of VUs. I am not an expert of UDP usage of sockets, but since all of this must happen on the same machine (unless I want to use k6 cloud), I suppose it will be impossible for any machine to support all of this. Am I right?

What do you generally do to simulate more VUs then the large machines can produce? Do you just write a script to start the same k6 script from multiple machines at the same time (without orchestrating anything)?

I am concerned about this and I hope I can get your honest opinion.
Thanks

Hi there, welcome to the forum!

I’m planning a load test on a system which has the VUs exchange UDP packets based on a scenario (with http calls as well).

That’s an interesting use case. However if you just need some central coordination between VUs, even across distributed k6 runs, Redis might be worth looking into. There are a couple of Redis extensions already, and technically this was possible even before xk6 by using Webdis.

So if that would work for your use case then you can avoid having to write support for UDP, though I agree that using xk6 would be the way to go in that case, since we do want to eventually support raw TCP/UDP packets.

One thing I’m not clear though is that you mention VUs exchanging UDP packets datagrams, but surely not directly and via some central server, otherwise you’d need to start a UDP server in k6, which is out of scope for what JS extensions are meant to be used. Not impossible, mind you, just likely very cumbersome.

Then I read the page about large tests, and I’m wondering if k6 is worth considering anymore as it does not support clustering.

As you can see on that page, there is some support for distributed execution. This currently only partitions the amount of VUs/iterations, but support for data segmentation is on the short-term roadmap as well.

Each of my VUs must be able to send or receive hundreds of small UDP packets per second

Are you sure you need direct communication between VUs though, i.e. being able to receive as well? Like I mentioned above, having a central coordination server like Redis is probably the easiest approach in this case since you wouldn’t need to start a UDP server in k6.

I must be able to simulate millions of VUs.

Are you also sure this is a hard requirement? In my experience whenever the topic of “millions of VUs” is brought up the actual requirements turn out to be much less, as there are few systems that can scale to handle millions of concurrent users (i.e. think FAANG scale). Sometimes a million requests per second is a high enough goal, yet a more realistic one to set than a million users.

Unless you’re absolutely certain of the scale you need, I would suggest initially setting a lower goal in mind, which can simplify the actual implementation efforts.

I am not an expert of UDP usage of sockets, but since all of this must happen on the same machine (unless I want to use k6 cloud), I suppose it will be impossible for any machine to support all of this. Am I right?

I’m not sure what the UDP limits are on different operating systems. On Linux TCP connections are limited by the amount of open file descriptors, configurable via ulimit -n. But since UDP is connectionless I imagine a limit would be much higher, if there is one at all.

See this OS fine tuning guide for some details, though most of it is related to TCP. Otherwise k6 can scale up to consume all resources available on a single machine, so you’d really be limited by the hardware you run it on.

What do you generally do to simulate more VUs then the large machines can produce? Do you just write a script to start the same k6 script from multiple machines at the same time (without orchestrating anything)?

I’m not sure we can divulge the secret sauce of k6 Cloud :wink:, but this is not far off from how it works, since k6 Cloud uses the same public release of k6. There is some orchestration on top of it, and we use the distributed execution functionality I mentioned above, but otherwise that’s the idea.

Hope this helps.

Thanks for the thorough answer!

I’ve thought about all of this and ended up writing an xk6 Go extension to send/receive UDP datagrams (to simplify things, I’ll let every VU be it’s own set of “clients” which will connect together). I’ve only tested with a single VU, but it seems to work fine.

My minimum requirements are indeed multiple millions of CCUs, so I’ll need to use an orchestrator. I’m looking into the k6 operator and only need to find out if it’s compatible with xk6 extensions. Would you know about that?