From my understanding of this (and your explanation) the problem is that datadog-agent once aggregates and calculates p95 and then the thing showed in the above is the average of those p95 that were calculated over the selected period. Which while probably useful enough for most people will likely never be exactly equal to the p95 k6 calculates which is over the whole period and without any aggregation (for now, we do plan on aggregating as the current behaviour is pretty … unoptimized and in practice leaks memory for big runs).
Apparently, also there is an issue with the datadog-agent so that the flush interval is configurable but it looks like it’s going nowhere 
.
To be honest it’s likely going to be very hard to make datadog flush only once for the whole test or something like that and also I would argue it will be … not useful as it will generate just 1 point for the p95(from my understanding, I might be completely wrong) for the whole test which likely isn’t what you would want.
I would argue the behaviour @Dilshan_Fernando reported is within what I would consider reasonable and likely what you will “want” from a practical perspective. While the original report was about so completely different values that at least to me it seems both unrelated and definitely a problem somewhere and given that k6 does not aggregate in this cases … I don’t think it’s k6 fault 
… but I am also a k6 developer so 
Hopefully @phaltoe has found a solution or the reason behind and will report it