Vus_max min less than max for constant-vus

[f_horsley]

I feel like I’m missing something here, but I’m using the constant-vus executor and the vus_max max/min values are different. From my understanding the constant-vus executor runs N numbers of vus for a given duration, so wouldn’t the vus_max min/max numbers be equivalent, as N number of VUs is created during init and only torn-down once the test has completed?

Does this suggest that one of the following? Either:

1. Insufficient VUs are being created before the test starts. In this case that would affect the test results, as would require time for more VUs to be created

2. VUs are idle during the test run, and are being cleaned up by K6. And then being re-created when needed?

   vus…: 275 min=0 max=275
   vus_max…: 275 min=40 max=275

[eyeveebee]

Hi @f_horsley

Welcome to the community forum

You are correct that the constant-vus executor maintains a constant number of virtual users (VUs) throughout the test duration. So ideally, the vus_max min and max values should be the same.

For example, if we run the example in the docs Constant VUs with 275 users:

import http from 'k6/http';
import { sleep } from 'k6';

export const options = {
    discardResponseBodies: true,
    scenarios: {
        contacts: {
            executor: 'constant-vus',
            vus: 275,
            duration: '30s',
        },
    },
};

export default function () {
    http.get('https://test.k6.io/contacts.php');
    // We're injecting a processing pause for illustrative purposes only!
    // Each iteration will be ~515ms, therefore ~2 iterations/second per VU maximum throughput.
    sleep(0.5);
}

The output will have the expected 275 vus, min and max:

     vus............................: 275    min=275      max=275
     vus_max........................: 275    min=275      max=275

In your example he vus_max min value is 40, indicating that the executor may start with a lower number of VUs and ramp up to 275 gradually. This could happen, for example, if your script has some heavy initialization or setup that takes time to complete. Without seeing the script it is difficult to say, though.

Can you share a (sanitized) script? Do you see any errors or warnings in the test output?

This will help provide better insights tailored to your case.

Cheers!

[f_horsley]

Hi thanks for such a quick reply!

I have no setup code, with the test script only running the main function. I’m using the following config

... // omitted for brevity
    my_api_test: {
      executor: 'constant-vus',
      startTime: `${vuInitTimeoutSecs}s`, // 1 second
      vus: VUsCount, // 275 VUs
      duration: `30s`,
      gracefulStop: `${loadTestGracefulStopSecs}s`, // 15 seconds
      tags: { type: 'loadtest' },
      exec: 'apiTest',
    },
...

How does K6 create the VUs before executing? I was expecting it to create all the VUs before starting the test code (and potentially fail and execute the run if it couldn’t), but by the sound of it they’re created in a ‘best effort’ manner so the test could start even though all the VUs haven’t been created yet?

Worth noting: I have a test harness that is repeating the K6 run n number of times. Is there a possibility that I’m not waiting long enough during the teardown, causing the next iteration init (VU creation) to be impacted as they is already resources allocated from the last iteration?

Thanks

[eyeveebee]

Hi @f_horsley

I was discussing this with @mstoykov and he points out that the values shown are actually the final values emitted. The vus and vus_max are emitted every 2 seconds. So it is somewhat expected that the value will be between 0 and the max value. In my example I was probably getting the right values due to the sleep(0.5);, as it’s an unusually consistent test.

Those values are then not super relevant, as they are the “last” values emitted.

To make this clearer:

1. k6 allocates VUs at the test start and there is no mid test allocation for constant-vus, as documented in the constat-vus executor
2. There is no “idle” time unless there is a sleep.

I hope this helps clarify, and thanks for posing the question, as it helped me learn how this works

[f_horsley]

hi @eyeveebe

Thanks Sounds like I’m good to use the vus_max value to plot how many VUs were used in the constant-vus executor.

I’m still not sure why the vus_min value is less than vus_max in my scenarion; the constant-vus executor is allocating vus_max VUs during init, so the fact that I’m seeing vus_min be lower suggests to me that something in my test is causing VUs to be towndown (resulting in the lower vus_min value)?

I’m a bit concerned that I’ve got the wrong assumptions about this executor; I thought that this test would create and maintain vus_max VUs from the start of the test, but the vus_min implies to me that at some point the API under test will have less load as there are fewer VUs running the apiTest() func.

I have idle time in my test, as I’m replicating a client polling an endpoint, which uses sleep to add a delay between requests. In this scenario, could the sleep(5) introduce “idle” time, leading to K6 tearing down VUs?

Sanitised test function:

export function apiTest() {

  // Get an access_token for this VU
  // AccessTokens are pre-fetched before the test to
  // prevent auth from impacting the test results
  const user: User = data[exec.vu.idInInstance];

  // poll activity for the length of the test
  // Believe this keeps the VU 'alive' for the length of the test (`duration`)??
  while (true) {
    const httpReqParams = { headers: {}, redirects: 0 };

    httpReqParams.headers['Authorization'] = `Bearer ${user.accessToken}`;

    const activityRes = http.get(`${targetCM}/api/activity`, httpReqParams);

    check(activityRes, {
      'check we fetched activity ok': (res) => res.status === 200,
    });

    // replicate standard UI poll delay
    sleep(5);
  }

}