Stream error: stream ID x; CANCEL

k6 support
#1

I run my scenarios locally from CLI.

In some of my scenarios, I get the following error for higher numbers of VU:

WARN[0036] Request Failed error="get \"https://<url>": stream error: stream ID 13; CANCEL"

It only seems to happen on scenarios with large number of parallel (batch()) requests. I start getting the error somewhere around 25 VUs (each at 6 parallel requests for majority of the scenario’s duration) and error rate steeply increases with additional VUs.

It seems to happen randomly for all kinds of requests, including static content.

I failed to replicate the error on scenarios with only serial requests (no batch()) - even at 2000 VUs I didn’t get the error once.

The stream ID always seems to be an odd number for some reason.

Here is a response object captured for one such failed request:

{
  "remote_ip": "<ip>",
  "remote_port": 443,
  "url": "https://<url>",
  "status": 0,
  "proto": "",
  "headers": {},
  "cookies": {},
  "body": null,
  "timings": {
    "duration": 16953.3841,
    "blocked": 0,
    "looking_up": 0,
    "connecting": 0,
    "tls_handshaking": 0,
    "sending": 0,
    "waiting": 16953.3841,
    "receiving": 0
  },
  "tls_version": "",
  "tls_cipher_suite": "",
  "ocsp": {
    "produced_at": 0,
    "this_update": 0,
    "next_update": 0,
    "revoked_at": 0,
    "revocation_reason": "",
    "status": ""
  },
  "error": "stream error: stream ID 29; CANCEL",
  "error_code": 1000,
  "request": {
    "method": "get",
    "url": "https://<url>",
    "headers": {
      "Accept-Encoding": [
        "gzip, deflate, br"
      ],
      "Accept-Language": [
        "en-US,en;q=0.9"
      ],
      "Sec-Fetch-Dest": [
        "empty"
      ],
      "Sec-Fetch-Mode": [
        "cors"
      ],
      "Sec-Fetch-Site": [
        "same-origin"
      ],
      "Accept": [
        "application/json, text/plain, */*"
      ],
      "Cookie": [
        "loginCookie-<hash>=<cookie value>"
      ],
      "User-Agent": [
        "k6/0.28.0 (https://k6.io/)"
      ]
    },
    "body": "",
    "cookies": {
      "loginCookie-<hash>": [
        {
          "name": "loginCooki-<hash>",
          "value": "<cookie value>",
          "replace": false
        }
      ]
    }
  }
}

I was unable to find anything about this error. Is it an error with K6 only, or does it indicate a problem with my server? How can I find out more about it?

#2

This is an HTTP/2 error, see https://tools.ietf.org/html/rfc7540#section-7:

CANCEL (0x8): Used by the endpoint to indicate that the stream is no longer needed.

I am not completely sure, but I think might mean your server is getting a bit overloaded? Or maybe it has a bug somewhere? :man_shrugging:

Generally, each VU in k6 is mostly independent, so normal non-batch requests from multiple VUs will use separate connections to the server. But with http.batch(), the VU will perform multiple requests simultaneously. And when you’re using HTTP/2, as your error suggests you are, these requests will likely be multiplexed on the same network connection. So this might be the reason you’re only seeing the error when you use http.batch().

It’s not impossible this is some bug in the Go standard HTTP library, on which k6 depends to make HTTP requests. Just to rule that out, can you share which k6 version are you using? If you’re using an older one, can you upgrade to the latest k6 v0.29.0 and confirm that the error persists? Also, which OS are you using and are you running k6 from docker or from the plain binary?

#3

Actually, this might be a k6 bug after all. I investigated things a bit, and it turns out we were depending on an old version of the Go HTTP/2 library. So it might be a bug that stems from that… :disappointed:

I made a note of that in https://github.com/loadimpact/k6/issues/548#issuecomment-731245076 and prioritized the issue to get rid of the extra dependency sooner, if possible. And I also created https://github.com/loadimpact/k6/pull/1734 to update the dependency, which we’ll merge in master soon. You should then be able to use the Docker master image, or build a k6 binary yourself, to see if you experience the same issue.

#4

The change was merged in master, can you try to use the master docker image and see if the problem disappears?