Similar Checks different failures

Grafana k6 OSS Support
1

Hello, I got recently test results where there was 3 checks included in script:

  1. Response is 201.
  2. Should contain body text
  3. Should contain body text (different text from other check)

And I cant figure out why the third check had 4 less failures. Also those 4 failures had status code 0 or 1050 as per k6 documentation. Could anyone perhaps explain why only third check did not fail 4 times? Because failed Requests was 1410, but third failed only 1406 times, but did have response Status 201 ?

image
image2342Ă—946 271 KB

3

Hi @sillymoomoo

Welcome back!

Could you share the script part of the checks? The checks are evaluated independently, so I hope the body text might give us some clues. Although if those were timeouts, I would not expect the body text to be a check.

You mention code 1050 on the ones that should have failed the 3rd check, am I understanding this correctly? Is that error code tagged to one of the requests where you see the second check fail and the 3rd pass?

Thanks in advance for sharing additional info.

Cheers!

5

Hello!

Thanks for your response, let me try providing more data.
This is the checks that are performed:

image
image878Ă—196 30.2 KB

And the code 1050 failed third check 4 times, but did not register those 4 requests as failed check with missing body content I believe, it still though that body was returned.

The test was ran on only 1 API Endpoint. All I saw from results in grafana that there were 4 http requests with error code 1050, and I am assuming that third check which has 4 less failures got that code :question:

As I understand CHECK should pass or fail if specific text was returned and it was in the body that was returned.

But this was impossible to figure out on my own, why third check had different amount of failures :roll_eyes:

Also here is the part how test was configured:

image
image608Ă—1302 101 KB

Thank you a lot for helping me out :slight_smile:

9

Hi @sillymoomoo,

Because you are accessing the body of a response that does not have a body you will get a null and then .includes will throw an exception - you likely can notice this in the console output.

The exception will faIl the check currently being executed, but will also stop the execution of the rest of the checks and will be “bubble up” as normal eventually stopping the execution of the current iteration if no try/catch is used.

I have opened an issue about it, but am not certain what hte fix should be as in this case we will run both and both will fail with an exception. Should we then throw the first one? the second one? neither? Some combination of the two?

Please if you have a particular opinion on this mention it in the issue.

Hope this helps you!

1 Like
11

Hello!

Well it is still kind of difficult to grasp why only third check “succeeded” or did not fail 4 times, as you would normally expect if API request fails/null/exception, all three checks should have same response :slight_smile:

12

Well it is still kind of difficult to grasp why only third check “succeeded”

it didn’t succeed it was not run at all.

The thing that happens is basically

console.log("check 1")
console.log("check 2")
if (some condition) {
  throw "some exception"
}
console.log("check 3")

So you do check 1 and then do check 2 (partially) and then you hit an exception.

Just as in normal JS execution that starts propagating backwards without executing the code after it.

In the issue I do give some alternatives on how it can behave, but I am not certain those are any better really :person_shrugging: . But you are definitely welcome to comment with your opinion on how this should work.

1 Like