Load testing for xk6-browser in the cloud

Hi there,
First of all this is an amazing tool you are building my blessings to you.
I wanted to know if the XK6.browser is available in the cloud so that I can perform like a 100.000 test at a time, to make a load balancing test.
Our page launches a WebGL Unity platform so it is important that we get responses in JS and it is run in a browser.
Thanks and kind regards

Hi @Manuel.
xk6-browser is solely a local CLI tool with support for only local development as of right now. It hasn’t been integrated with the k6 cloud yet as it is still in the development phase.

Thank you very much for your quick response,
Can you tell me if this integration is in the roadmap and for when?
How many browser you think I can launch on my local machine? I have an last year apple Macbook pro
Thanks and kind regards

I think I saw it somewhere that they plan to add integration for cloud too, but not sure when. Please check here if you can find any issues regarding cloud integration.

As for the limit of browsers you can start on your local machine, I’m not really sure, you have to play a bit with some numbers and see how it works. Personally I have done some simple tests with ~20 chrome instances at the same time → Macbook pro i7 2019. I think you should be able to go with higher numbers and see how it goes.

1 Like

Hi Manuel, welcome to the forum! We’re glad you’re enjoying the tool :slight_smile:

Like @rdt.one mentioned, xk6-browser is not available in the k6 Cloud yet. We have some ongoing plans to do that soon, but we can’t provide an ETA yet.

Your use case for WebGL is interesting. How has your experience been so far testing the extension on your local machine?

Since WebGL will require a discrete GPU to be performant, this complicates things substantially:

  • the Cloud instances will likely be more expensive than regular ones where a GPU is not a requirement.

  • How would multiple browser processes using WebGL on the same GPU work? Is that supported, and if so, what is the performance impact? I guess it would depend on the app itself, but all of this might raise resource requirements–and thus cost–significantly.

I would add that WebGL and GPU support in the Cloud is a step beyond making xk6-browser usable on regular sites. So I really can’t say when or if that will be prioritized.

Regarding your 100.000 (VUs?) point, note that xk6-browser is not really meant to be used for load testing, but for functional/E2E/frontend testing. This is simply because a browser process is launched per VU, which is much heavier on system resources than k6’s virtual users.

To give you an idea, testing a relatively simple site like https://test.k6.io/ launches a Chromium process that consumes around 200MB RAM on my machine, and doesn’t use much CPU. Note that this usage is in addition to regular k6 memory usage, which might be in the tens of MB per VU, depending on your JS script. So launching thousands of xk6-browser VUs on a single machine will likely not be possible, unless you have a very powerful machine (think hundreds of GBs or even TB+ of RAM).

We’ve yet to measure the performance and system requirements needed for regular xk6-browser tests, let alone WebGL ones, but my point is that you shouldn’t expect to use the tool for load testing, but for functional/E2E/frontend testing instead. There’s nothing practically preventing you from doing that, it’s just that it will be very costly to deploy, either locally or in the Cloud.

Let us know if you have any more questions, and we’d love to hear your feedback so far. Cheers!

First of all, thank you so much for your support!!
So far I am not being able to make it work not even for a single browser :frowning:
I can see that the memory required for the test would be huge but even though I have to try it, I expect that we can pay it maybe only once at least :slight_smile:
What would you recommend me for load testing on a WebGL that fits my needs?
I am having this error:
ERRO[0001] a panic occurred in VU code but was caught: runtime error: invalid memory address or nil pointer dereference executor=per-vu-iterations scenario=default
Maybe I have to reserve more memory or something?
Thank you so much and kind regards

That error certainly shouldn’t happen. Please open a new GitHub issue that includes the xk6-browser version you’re using, your OS and version, the full error (including the Go stack trace) and your script if possible. If your site is available publicly that would help as well, but I suppose we can use any WebGL site to test this. Note that it’s important that we’re able to reproduce the issue in order to fix it, so any relevant information you provide is helpful. Thanks!