How to structure code for multiple test types

Grafana k6 OSS Support
1

Goal 1: For a particular test case, I want to version test types (smoke, stress, …) independently
Goal 2: Pass a list of test types to execute via an env variable (or only one).

I implemented this using the “scenarios” feature, but I don’t think it’s good, because I can’t execute scenarios sequentially and there is nothing like “nested scenarios”, in which I can specify test type (parent scenarios) and the actually test scenarios for the test type.

if (__ENV.TEST_TYPE && __ENV.TEST_TYPE.includes("stress")) {
scenarios.stress = {
    executor: 'constant-arrival-rate',
    rate: 25,
    timeUnit: '1s',
    duration: '1m',
    preAllocatedVUs: 50,
    maxVUs: 100,
    }
} else {
    scenarios.smoke = {
        vus: 1,
        duration: '30s',
        executor: 'constant-vus',
    }
}

export let options = {
scenarios: scenarios,
thresholds: {
    'http_req_duration{scenario:smoke}': ['p(99)<1500'],
    'checks{scenario:smoke}': ['rate=1'],
    'groupDuration{groupName:userFlowHomepage,scenario:smoke}': ['p(95) < 500'],
    'groupDuration{groupName:findAllWelfareEntityByCategory,scenario:smoke}': ['p(95) < 300'],
    'groupDuration{groupName:findWelfareEntity,scenario:smoke}': ['p(95) < 300'],

    'http_req_duration{scenario:stress}': ['p(90)<1500'],
    'checks{scenario:stress}': ['rate=0.9'],
    'groupDuration{groupName:userFlowHomepage,scenario:stress}': ['p(90) < 3000'],
    'groupDuration{groupName:findAllWelfareEntityByCategory,scenario:stress}': ['p(90) < 1500'],
    'groupDuration{groupName:findWelfareEntity,scenario:stress}': ['p(90) < 1500'],
}}

As you can see, I have only one “thresholds” that discriminate via the scenario tag. With this arrangement, I can version all the scenarios in different js file (I can do that even for thresholds I guess).

Should I drop the scenario feature and use plain options and remove that “multiple tags selector” from thresholds?

What are the best practices? I couldn’t find anything about how I should structure my code for different types of tests.

2

Unfortunately it’s difficult to suggest best practices when there is such a huge variety of use cases… :disappointed:

Regarding your specific use case: if I understand it correctly, you don’t have a difference in the test script between the smoke and stress test, only the executor and thresholds options are different? Then you might want to pass the test type with an environment variable (e.g. k6 run --env TEST_TYPE=stress, k6 run --env TEST_TYPE=smoke) and just load the options for that specific test type somewhat like this:

export let options = JSON.parse(open(__ENV.TEST_TYPE));

In general, I’d consider adding thresholds for sub-metrics that you know won’t be encountered (e.g. 'checks{scenario:smoke}' when you’re running the stress test) a somewhat bad practice, so this way you can avoid it.

Edit: forgot to mention that you could also pass custom JSON configs, see Options reference

2 Likes
3

Thank you for your reply, I think I’ll pick pass custom JSON configs!

4

Can please you share your solution after passing custom JSON configs? I have similar problem statement like you do. Thanks in advance.