Basic k6 configuration

Hello,

We have a consistent number of traffic on our website. Based on this article we have determined the concurrent number of users be 50. I have generated a js script for load testing. The stages look like the following:

stages: [
	    { duration: "1m", target: 50 },
            { duration: "3m", target: 50 },
            { duration: "1m", target: 0 },
	],

The problem is even with the expected concurrent users the 95% of response time is greater than 12s. Another thing is we have not got any high CPU notification to date due to high traffic but when doing the load test, the CPU load increases up to 80%. are we missing anything here?

What should be the correct duration for testing? Are the above stages correct? Do I need to change the concurrent users?

Hi @demon_js,
welcome to the community forum :wave:

The stages per-se are fine, the k6 documentation has some explanation about different types of tests that maybe can give you more context

However, for understanding, if the stages fit your specific case we need more info. Can you share more scripts’ details (with anonymized data), please? Knowing the protocol you are using and the invoked functions could be helpful. Also the other metrics you’re getting from k6 results.

k6 can’t directly influence the response time of your requests except if you’re running it directly from the same instance consuming and creating resources concurrency with your under-tested service.

So if the previous is respected but you’re detecting a high response time then here are some points to check:

  • Is the networking between k6 and the service accessible, stable and reliable?
  • Is the service under-tested a dedicated/isolated environment or does it receive at the same time traffic from other sources?
  • Is the rate computed using the article relative to the same scenarios you are testing?
  • Is your system hit any sort of trouble when the load is generated?
  • As suggested from the Note about performance thresholds, is the test able to run with a smaller amount of traffic?

Hey @codebien,

Thank you for the reply. I have read about the load test thoroughly. The documentation describes the different types of tests with examples, and it is easy to understand. Let me know if I am missing anything though.

I have generated a simple script from the k6 website:

import { sleep, check, group } from 'k6';
import http from 'k6/http';

export const options = {
	stages: [
		{ target: 50, duration: '1m' }, // 50 virtual users (VUs)
		{ target: 50, duration: '3m' },
		{ target: 0, duration: '1m' },
	],
	thresholds: {
		// Requirement to pass the tests
		http_req_duration: [
			'p(95)<3000', // 95% of response times must be below 3s
			// { threshold: "p(95)>=5000", abortOnFail: true },
		// ],
		http_req_failed: [
			'rate<0.01', // Error rate should be less than 1%
			{ threshold: "rate>=0.05", abortOnFail: true },
		],
	},
};

export default function main() {
	let response;

	group( 'Homepage group', function () {
		// Homepage
		response = http.get( 'https://example.com/' );
		check( response, {
			'status equals 200': ( response ) => response.status.toString() === '200',
			'body contains My website': (
				response
			) =>
				response.body.includes(
					'My website'
				),
		} );
		sleep( 3 );
	} );

	group( 'abc page', function () {
		// abc page
		response = http.get( 'https://example.com/abc' );
		check( response, {
			'status equals 200': ( response ) => response.status.toString() === '200',
			'body contains abc page': ( response ) =>
				response.body.includes( 'abc page' ),
		} );
		sleep( 2 );
	} );

	group( 'Blog page', function () {
		// Blog
		response = http.get( 'https://example.com/blog' );
		check( response, {
			'status equals 200': ( response ) => response.status.toString() === '200',
			'body contains blog title': ( response ) =>
				response.body.includes( 'blog title' ),
		} );
		sleep( 2 );
	} );
}

I have ran the tests from my local machine. The 95% response time is 8.51s (I probably noted “max”), here is the result.

 checks.........................: 100.00% ✓ 3408     ✗ 0
 data_received..................: 214 MB  705 kB/s
 data_sent......................: 651 kB  2.1 kB/s
 group_duration.................: avg=7.27s    min=3.14s    med=7.36s    max=15.36s   p(90)=10.71s   p(95)=11.53s
 http_req_blocked...............: avg=13.2ms   min=0s       med=1µs      max=1.82s    p(90)=2µs      p(95)=2µs
 http_req_connecting............: avg=6.59ms   min=0s       med=0s       max=1.24s    p(90)=0s       p(95)=0s
✗ http_req_duration..............: avg=3.68s    min=235.32ms med=3.45s   max=12.36s   p(90)=7.65s    p(95)=8.51s
   { expected_response:true }...: avg=3.68s    min=235.32ms med=3.45s    max=12.36s   p(90)=7.65s    p(95)=8.51s
✓ http_req_failed................: 0.00%   ✓ 0        ✗ 2272
 http_req_receiving.............: avg=440.4ms  min=27µs     med=502.04ms max=3.29s    p(90)=791.97ms p(95)=1.21s
 http_req_sending...............: avg=195.33µs min=23µs     med=164µs    max=1.67ms   p(90)=341.9µs  p(95)=394.44µs
 http_req_tls_handshaking.......: avg=5.97ms   min=0s       med=0s       max=516.03ms p(90)=0s       p(95)=0s
 http_req_waiting...............: avg=3.24s    min=235.06ms med=2.94s    max=11.42s   p(90)=6.86s    p(95)=7.68s
 http_reqs......................: 2272    7.492106/s
 iteration_duration.............: avg=21.81s   min=11.11s   med=22.76s   max=32.65s   p(90)=26.95s   p(95)=28.18s
 iterations.....................: 568     1.873026/s
 vus............................: 1       min=1      max=50
 vus_max........................: 50      min=50     max=50

I ran the tests with 20 users but it resulted in high CPU load. Do you suggest to gradually increase the concurrent users as following?

stages: [
	{ target: 10, duration: '30s' }, 
	{ target: 20, duration: '30s' },
	{ target: 30, duration: '30s' },
	{ target: 40, duration: '1m' },
	{ target: 50, duration: '1m' },
	{ target: 50, duration: '5m' },
	{ target: 0,  duration: '1m' },
],

Lets say if the system behaves okay when I set concurrent users to 10 and increases CPU load if I increase concurrent users, then what is the point of calculating the users from analytics?

Hi @demon_js,

apparently, some requests are taking longer on your server. It could be expected considering the amount of CPU you’re seeing. Have you checked metrics for your service? Are they reflecting what k6 is reporting?

No, I meant you could try to simplify the total load, starting from 1 and gradually increase finding the breaking point:

k6 run --vus 1 --duration 1m script.js

Assuming that the math used for computing the VUs was right, then it means that the load scripted is not reproducing the same journey described by analytics. (e.g. is the sleep value accurate for simulating your page loading?)

The stages options is a shortcut for using the constant-vus executor that executes as many iterations as possible. Is the rate generated aligned with what you expected?

Hello @codebien ,

I think I will do what you have suggested, gradually increase the vus and check the breaking point. Another question, what should we consider the breaking point, > 80% CPU utilization? or is it when the reponse code is not 200?

Another question, what should we consider the breaking point, > 80% CPU utilization? or is it when the reponse code is not 200?

When you see failing one of your k6’s checks and/or thresholds. It’s the way to detect that something is going in an unexpected way.

Thanks. I will do as you advise.