Forbidden error with error code 1000 for k6 run from Codefresh

Grafana k6 OSS Support
1

Hi,

We are trying to run a load test using k6 from Codefresh and are getting a Forbidden error with error code 1000

k6 version - 0.39

We tried a simple test as documented at Running k6 and that worked fine.

While trying a HTTP GET call to our server using mutual authentication we are getting the forbidden error

k6 run details as below -

data_received..............: 35 kB   3.2 kB/s                                                                         
data_sent..................: 1.2 kB  110 B/s                                                                          
http_req_blocked...........: avg=0s       min=0s       med=0s       max=0s      p(90)=0s       p(95)=0s               
http_req_connecting........: avg=862.43µs min=748.74µs med=876.67µs max=1.03ms  p(90)=972.86µs p(95)=1ms              
http_req_duration..........: avg=0s       min=0s       med=0s       max=0s      p(90)=0s       p(95)=0s               
http_req_failed............: 100.00% ✓ 10       ✗ 0                                                                   
http_req_receiving.........: avg=0s       min=0s       med=0s       max=0s      p(90)=0s       p(95)=0s               
http_req_sending...........: avg=0s       min=0s       med=0s       max=0s      p(90)=0s       p(95)=0s               
http_req_tls_handshaking...: avg=0s       min=0s       med=0s       max=0s      p(90)=0s       p(95)=0s               
http_req_waiting...........: avg=0s       min=0s       med=0s       max=0s      p(90)=0s       p(95)=0s               
http_reqs..................: 10      0.908904/s                                                                       
iteration_duration.........: avg=3.7ms    min=2.38ms   med=2.71ms   max=12.81ms p(90)=4.15ms   p(95)=8.48ms           
iterations.................: 10      0.908904/s                                                                       
vus........................: 5       min=5      max=5                                                                 
vus_max....................: 5       min=5      max=5

Console log details as below
(server referred as https://host:port/test in the details) -

INFO[0011] {"remote_ip":"","remote_port":0,"url":"https://host:port/test","status":0,"status_text":"",
"proto":"","headers":{},"cookies":{},"body":null,"timings":{"duration":0,"blocked":0,"looking_up":0,"connecting":0.789388,"tls_handshaking":0,"sending":0,"waiting":0,"receiving":0},"tls_version":"","tls_cipher_suite":"","ocsp":{"produced_at":0,"this_update":0,"next_update":0,"revoked_at":0,"revocation_reason":"","status":""},**"error":"Forbidden","error_code":1000**,"request":{"method":"GET","url":"https://host:port/test","headers":{"User-Agent":["k6/0.39.0 (https://k6.io/)"]},"body":"","cookies":{}}}  source=console

The same script runs fine from our local machines.

Is it possible to enable a trace to get more details about the issue

Any pointers on resolving the issue would be extremely helpful

3

Hi @Shaila,
welcome to the community forum :tada:

The documentation has a page for HTTP debugging, take a look.

Let me know if it helps.

5

Thankyou @codebien,

we have tried the HTTP debug option which provides the below information -
WARN[0001] error getting terminal size error=“”

GET /test HTTP/1.1
Host: host:port
User-Agent: k6/0.39.0 (https://k6.io/)
Content-Length: 17
Content-Type: application/x-www-form-urlencoded
Accept-Encoding: gzip

WARN[0011] Request Failed error=“Get "https://host:port/test": Forbidden”

INFO[0011] {“remote_ip”:“”,“remote_port”:0,“url”:“https://host:port/test”,“status”:0,“status_text”:“”,
“proto”:“”,“headers”:{},“cookies”:{},“body”:null,“timings”:{“duration”:0,“blocked”:0,“looking_up”:0,“connecting”:0.786378,“tls_handshaking”:0,“sending”:0,“waiting”:0,“receiving”:0},“tls_version”:“”,“tls_cipher_suite”:“”,“ocsp”:{“produced_at”:0,“this_update”:0,“next_update”:0,“revoked_at”:0,“revocation_reason”:“”,“status”:“”},“error”:“Forbidden”,“error_code”:1000,“request”:{“method”:“GET”,“url”:“https://host:port/test”,“headers”:{“User-Agent”:[“k6/0.39.0 (https://k6.io/)”]},“body”:“”,“cookies”:{}}} source=console ‘’

How can we get more information on the reason for the Forbidden error

The same call runs fine with a HTTP 200 response while we run via curl command from Codefresh but does not work from the k6 script.

6

Hi @Shaila,
it seems the current OS you are using denies access to the network stack from the k6 process. I guess you have to resolve this issue on Codefresh providing more permissions to the k6 process.

Which OS is using your Codefresh instance?

Let me know.

7

Hi @codebien
We are using an alpine base image and have installed the k6 linux (amd64)pkg on top of that.

We are able to run a sample script documented at Running k6 and that executes fine.

8

Can you post an anonymized version of your script so we can check for eventual anomalies?

9

From this message this looks like the OS just tells us forbidden - maybe it is due k6 trying to open a connection on a host/port combo it doesn’t like. Maybe it’s for some other reason.

But this doesn’t seem like a 403 status code at all. You can see how there is no actual response from the server and how the status code is 0 not 403 and the error is just Forbidden. If you do an actual 403, against httpbin for example, you will see that it looks completely differently.

I am not certain that a script will help us as this seems like a system issue. I would recommend looking into logs - maybe potentially strace-ing k6 … although that will likely be very noisy.

10

Hi @codebien

please find below the details of the script -

import http from "k6/http";
import encoding from 'k6/encoding';

const CERT = `-----BEGIN...`;

const KEY  = `-----BEGIN... `;

// set status codes to be regarded as successful responses 
  http.setResponseCallback(http.expectedStatuses({min: 200, max: 300}));

//Create variables 


  var baseurl = 'https://host:port';
  var url1 = baseurl + '/test'


// Options. you could also pass these as args at the time of executing the script
export let options = {
  // https://k6.io/docs/using-k6/options/
  discardResponseBodies: true,
  setupTimeout: '10s',
  tlsVersion: 'tls1.2',
  thresholds: {
    //https://k6.io/docs/using-k6/thresholds
    http_req_failed: ['rate<0.01'],    // http errors should be less than 1%
    http_req_duration: ['p(95)<300'],  // 95% of requests should be below 300ms
},
  scenarios: {
      //https://k6.io/docs/using-k6/scenarios/
 ramping_rate: {
        executor: 'ramping-arrival-rate',
        startRate: 1,
        timeUnit: '1s',
        preAllocatedVUs: 5,
        maxVUs: 10,
        stages: [
	    { target: 1, duration: '1s' },
        { target: 1, duration: '10s' },
		],
      },
    },
tlsAuth: [
    {
	cert: CERT,
	key : KEY,    
    },
	],	
};

// Main function for vu
export default function() {
    let res = http.batch([
      ['GET', url1,],
    ]);
     for (let i = 0; i < 1; i++) {
       let element = res[i];
       console.log('logtime: ' + new Date().toISOString() + ' URL: ' 
           + ' status: ' + String(res[i].status)  
           + ' respTime: ' + String(res[i].timings.duration) + 'ms' 
           + ' reqSending: ' +  String(res[i].timings.sending) + 'ms' 
           + ' reqWaiting: ' +  String(res[i].timings.waiting) + 'ms' 
           + ' reqRecieving: ' +  String(res[i].timings.receiving) + 'ms' 
           + ' reqBlocked: ' +  String(res[i].timings.blocked) + 'ms' 
           + ' reqConnecting: ' +  String(res[i].timings.connecting) + 'ms' 
           + ' reqTLSHandshake: ' +  String(res[i].timings.tls_handshaking) + 'ms'  
           + ' "TLSCipherUsed": ' +  String(res[i].tls_cipher_suite) + ',' 
           + ' "TLSVersion": ' +  String(res[i].tls_version) + ',' 
           + ' "remoteIP": ' + String(res[i].remote_ip) + ','  
           + ' "remotePort": ' + String(res[i].remote_port) 
       );}  
  };
11

You can replace the big console.log at the end with just console.log(JSON.stringify(res[i], nil, " "))

But I don’t see anything that will make k6 behave like it did, so I do still think you need to look into logs and maybe strace to see the exact error that is being returned. (I do recommend running only 1 user for the strace)