Project

General

Profile

Actions

Feature #3483

open

Summarize results of performance testing to date

Added by Jeff Burke over 8 years ago. Updated about 8 years ago.

Status:
Resolved
Priority:
Normal
Assignee:
-
Category:
-
Start date:
02/28/2016
Due date:
% Done:

100%

Estimated time:

Description

Summarize informally and share the results of summer performance testing as they may inform the current tests of NDN-RTC.

Actions #1

Updated by Peter Gusev over 8 years ago

  • Status changed from New to In Progress

1-to-1 over isolated testbed

Our tests targeted improving NDN-RTC code and its' performance. We ran tests on isolated testbed 1-to-1, 5 min, audio, video, audio_video, video streams of 1000Kbit/s, no stream switching.
Rebufferings number (stream interruption) was the main metric we targeted to improve - there should be no interruptions during a test run.

We eventually improved NDN-RTC code through bugfixing, refactoring and fine-tuning of the estimators and achieved non-interruptible streaming in our tests. Alongside, we measured chasing times: these are the times for streams bootstrapping (chasing): video - 1-1.5sec, audio - 0.5sec.

1-way fetching multihop over testbed

Testbed tests didn't give any prominent results as the majority didn't even started due to failed discovery (which failed due to incorrect hub configurations or for other unexplained reasons). For those hubs that allowed discovery, results were not satisfactory (highly depended on intermediate hubs):

  • 0 hop test: 2 interruptions/OK
  • 1 hop test: ~8 interruptions/DECENT
  • 2 hop test: ~5 interruptions (BYU-CSU-REMAP, CAIDA-UCLA-REMAP), ~50 interruptions/FAIL (KISTI-CSU-REMAP, MICH-CSU-REMAP, UIUC-CSU-REMAP, UM-UA-REMAP)
  • 3 hop tests: 5 interruptions (LIP6-MICH-CSU-REMAP), ~50 interruptions/FAIL (NEU-MICH-CSU-REMAP, WU-UIUC-CSU-REMAP, ANYANG-WASEDA-UA-REMAP)

1-to-1 over testbed

1-to-1 tests over the testbed varied depending on the hub. On average, 2-4 interruptions observed during 5min tests. These were problematic hubs (dated August 19 2015) where consumer couldn't pick up fetching:

  • MEMPHIS (~50 interruptions);
  • NEU (~12 interruptions);
  • UIUC (~50 interruptions);
  • WU (~25 interruptions).
Actions #2

Updated by Peter Gusev over 8 years ago

  • Status changed from In Progress to Feedback
Actions #3

Updated by Peter Gusev about 8 years ago

  • Status changed from Feedback to Resolved
  • % Done changed from 0 to 100
Actions

Also available in: Atom PDF