Feature #3483
openSummarize results of performance testing to date
100%
Description
Summarize informally and share the results of summer performance testing as they may inform the current tests of NDN-RTC.
Updated by Peter Gusev over 8 years ago
- Status changed from New to In Progress
1-to-1 over isolated testbed¶
Our tests targeted improving NDN-RTC code and its' performance. We ran tests on isolated testbed 1-to-1, 5 min, audio, video, audio_video, video streams of 1000Kbit/s, no stream switching.
Rebufferings number (stream interruption) was the main metric we targeted to improve - there should be no interruptions during a test run.
We eventually improved NDN-RTC code through bugfixing, refactoring and fine-tuning of the estimators and achieved non-interruptible streaming in our tests. Alongside, we measured chasing times: these are the times for streams bootstrapping (chasing): video - 1-1.5sec, audio - 0.5sec.
1-way fetching multihop over testbed¶
Testbed tests didn't give any prominent results as the majority didn't even started due to failed discovery (which failed due to incorrect hub configurations or for other unexplained reasons). For those hubs that allowed discovery, results were not satisfactory (highly depended on intermediate hubs):
- 0 hop test: 2 interruptions/OK
- 1 hop test: ~8 interruptions/DECENT
- 2 hop test: ~5 interruptions (BYU-CSU-REMAP, CAIDA-UCLA-REMAP), ~50 interruptions/FAIL (KISTI-CSU-REMAP, MICH-CSU-REMAP, UIUC-CSU-REMAP, UM-UA-REMAP)
- 3 hop tests: 5 interruptions (LIP6-MICH-CSU-REMAP), ~50 interruptions/FAIL (NEU-MICH-CSU-REMAP, WU-UIUC-CSU-REMAP, ANYANG-WASEDA-UA-REMAP)
1-to-1 over testbed¶
1-to-1 tests over the testbed varied depending on the hub. On average, 2-4 interruptions observed during 5min tests. These were problematic hubs (dated August 19 2015) where consumer couldn't pick up fetching:
- MEMPHIS (~50 interruptions);
- NEU (~12 interruptions);
- UIUC (~50 interruptions);
- WU (~25 interruptions).
Updated by Peter Gusev over 8 years ago
- Status changed from In Progress to Feedback
Updated by Peter Gusev about 8 years ago
- Status changed from Feedback to Resolved
- % Done changed from 0 to 100