--summary-export command

Hello everyone, I am using the --summary-export command to export the test results to one file, but checking the thresholds metric that I was added, I found that when the threshold is being satisfied in the json the threshold is marked as false

    "create_identity_successful_requests": {
            "fails": 0,
            "passes": 34,
            "thresholds": {
                "rate>0.90": false
            },
            "value": 0
        }

and true for when it is not satisfied

    "associate_payment_credential_successful_requests": {
            "fails": 33,
            "passes": 0,
            "thresholds": {
                "rate>0.90": true
            },
            "value": 0
        }

this should be true when is satisfied and false when not.

Hi @joseray20,

Apparently at the time this was implemented nobody noticed that this will be ambiguous, but the current logic is that the Boolean answer the question “has the threshold been exceeded?”. So if true, it has failed :).

This is unlikely to change as it is a breaking change, and anyone who has been using it will now have to change how it is used. We are going to thoroughly document in the coming days and hopefully that will clear any ambiguity or misunderstanding of the output.

1 Like