SSE stream optimization

[szkiba]

On the SSE stream, the snapshot and cumulative events are displayed as JSON objects, where the metric names are the property names. The property values are also JSON objects, in which the property names are the names of the aggregation function (avg, min, max, etc.). This two-level object hierarchy has not caused any problems so far, the amount of data has been acceptable.

Currently, aggregate counting per tag is still supported, but by default it is only allowed for the group tag. It would be necessary to turn on the aggregate calculation for the url and name tags as well (per URL graphs, summary, etc.). With the current SSE stream format, this would significantly increase the amount of data. This would have a direct impact on the k6 memory requirement, so it is a problem.

Based on measurements, it can be concluded that using a two-level array structure instead of the two-level object structure used in snapshot and cumulative events significantly reduces the size of the events and thus also the memory requirement.

For example, for the scripts/demo/demo.js test script, the size of the SSE stream is reduced by 65%. With url and name tags enabled, the new format results in a 10% size increase compared to the original size (without these tags). That is, the new format enables aggregate calculation based on url and name tags, which is a prerequisite for several requested features.

Therefore, instead of the two-level object structure, the use of a two-level array structure must be implemented in the snapshot and cumulative events. The packages/model package can hide SSE stream changes from the rest of the dashboard UI.

Related issues:

• Adding a URL table panel to the timings tab #98
• Dashboards representing individual transactions' metrics rather than TOTALs #160