ndnrtc

OI dashboards can be found here.

Grafana is used as an OI tool, InfluxDB is used as a time-series data storage.

Grafana is not designed for using it as an analysis tool for occasional test runs. It is designed as visualization&dig-and-discover tool for continuous metrics gathered from remote services. Consequently, I had to come up with some usage pattern in order to use Grafana with NDN-RTC large-scale testing.

Key concepts:¶

• individual tests are combined into test-groups
• each test has a number and ID - date-time string
• test group is defined by tester, who launches test scripts;
  • this is typically a string containing meaningful identificator for the tests, i.e. "version0.1.1-beta", "NFD-fixed-strategy-issue", etc.
• test id and test group attached to each metric as tags

Usage pattern:¶

Analyst searches for certain test group using "Test Sets Overview" dashboard:

• first, analyst must choose time period that covers time when tests were taken (otherwise, data will not be retrieved);
• second, analyst must choose value for the "Test group" variable from the drop-down list (if value is not there, reload the page);
• if nothing is visualized on the graph - make sure time interval is correct (it doesn't need to be exact though);
• once data appears on the graph, analyst should be able to see differently colored dots for each test runs within test set:
  • from here - analyst should narrow down the time range on the graph that corresponds to the test she wants to see;
• once test data is narrowed (graph does not mean anything, it's just a visualization of the duration of the test), analyst must use hyperlink on top of the graph to jump to "Test Run Overview (NEW)" dashboard;
• from "Test Run Overview (NEW)" dashboard, analyst is able to jump to other dashboards using the same principle - hyperlinks next to the graphs' captions.