I’d like to be able to share some scripting between setup() and execution.
I’m trying to set up a repo where we have a set of stories that consume a bunch of library functions that behave reasonably either in setup() code or during the default execution (when the VUs are running the actual load test). On the face of it this seems easy, but error handling doesn’t make a lot of sense without completely abstracting what’s built in to k6 as during execution you want to just check to mark the failure and move on but during setup() if something fails, we’re losing critical structure we need for the test to make sense so we want a proper failure.
The other problem I’ve run in to is is that throw new Error(...) in setup() code doesn’t give a useful stack trace.
Has anyone else tried to share code in multiple modes and solved this already?
Looked into the stacktraces as I am currently working on upgrading babel and looked at sourcemaps briefly around it.
They were previously enabled and apparently added stacked traces but have since been disabled for speed reasons. Unfortunately just setting that flag to true doesn’t fix it for me
So can you open an issue for this and another one for the check issue.
I don’t have any good suggestion at this moment … I suppose you can differentiate between setup/teardown and default through checking for if __ITER is undefined ?
Any suggestion on how to make this better are welcome
Sorry for the slow reply, I think you read my mind about finding a way to figure out whether we were in setup/teardown as I realised afterwards I hadn’t asked that. Although checking __ITER doesn’t work for me, checking __VU === 0 does.
I ended up with this for now:
/**
* Define our own check() so that we can automatically change the exit code for
* failing checks in CI.
*
* Make sure to import this check in top level scripts with:
* import { check, thresholds } from "./lib/check.js";
* export let options = { thresholds: thresholds };
* so that the thresholds get configured.
*/
import { check as k6check } from "k6";
import { Rate } from "k6/metrics";
var failedChecks = new Rate("failed checks");
// Configure thresholds to exit non-zero if any checks fails, but only if we're
// in CI.
// GitLab CI at least will always define the CI env var, most other CIs probably
// do too.
export let thresholds = {};
if (__ENV.CI) {
thresholds = {
"failed checks": ["rate==0"],
}
}
export function check(data, checks) {
for (var check in checks) {
if (k6check(data, { [check]: checks[check] })) {
continue;
}
failedChecks.add(1);
// if __VU == 0 then we are in setup or teardown so onlya throw will get
// through
if (__VU !== 0) {
continue;
}
throw new Error("Failed check during setup: " + check);
}
};
Stack traces would still be nice, but it’s a start.
