Task #3526
openMarch 10 NDN-RTC test
80%
Description
Tests on March 9th demonstrated poor performance in multi-consumer scenarios for audio conferencing.
There is a need to narrow down testing and understand issues causing poor performance (choppy audio).
Files
Updated by Peter Gusev over 8 years ago
The following test set was proposed:
topology:
1 producer connected to REMAP, 2 consumers connected elsewhere on the testbed (among UCLA,CSU,WU,REMAP hubs)general description:
we’d like to explore how multiple consumers affect each other when fetching from the same producer. for that, one consumer will start fetching from producer, after some time (30sec) another consumer will start fetching from the same producer and we’ll see how this affects audio for both (does it affect at all). we will do several rounds of this test with one consumer connecting to different hubs on the testbed. before multi-consumer test, we’ll check that audio works decently for both consumers in 1-to-1 scenario.detailed steps:
- producer P publishes audio stream; connected to REMAP.
- audio is checked for each consumer separately:
- 1/ consumer C1 connects to WU and fetches audio for 30-60sec from P. confirm that audio works decently. consumer C1 stops fetching. report to chat.
- 2/ consumer C2 connects to hub and fetches audio for 30-60sec from P. confirm that audio works decently. consumer C2 stops fetching. report to chat.
- audio is checked for both consumers:
- 3/ consumer C1 connects to hub and fetches audio from P. after 30sec, consumer connects to hub and starts fetching audio from P. observe audio quality for both consumers for 60-90sec. report to chat
tests set:
I. P on REMAP, C1 on WU, C2 on WU¶
- C1 <-- P for 30sec (C1 fetches form P for 30 sec and stops)
- C2 <-- P for 30sec (C2 fetches from P for 30 sec and stops)
- C1 <-- P for 30sec + C2 <— P for 60-90sec (C1 fetches from P for 30 sec, then C2 starts fetching form P for another 60-90sec, then both stop)
II. P on REMAP, C1 on WU, C2 on CSU¶
- C1 <-- P for 30sec (C1 fetches form P for 30 sec and stops)
- C2 <-- P for 30sec (C2 fetches from P for 30 sec and stops)
- C1 <-- P for 30sec + C2 <— P for 60-90sec (C1 fetches from P for 30 sec, then C2 starts fetching form P for another 60-90sec, then both stop)
III. P on REMAP, C1 on WU, C2 on UCLA¶
- C1 <-- P for 30sec (C1 fetches form P for 30 sec)
- C2 <-- P for 30sec (C2 fetches from P for 30 sec)
- C1 <-- P for 30sec + C2 <— P for 60-90sec (C1 fetches from P for 30 sec, then C2 starts fetching form P for another 60-90sec, then both stop)
IV. P on REMAP, C1 on WU, C2 on REMAP¶
- C1 <-- P for 30sec (C1 fetches form P for 30 sec)
- C2 <-- P for 30sec (C2 fetches from P for 30 sec)
- C1 <-- P for 30sec + C2 <— P for 60-90sec (C1 fetches from P for 30 sec, then C2 starts fetching form P for another 60-90sec, then both stop)
Updated by Peter Gusev over 8 years ago
- Status changed from New to In Progress
- % Done changed from 0 to 60
Test results: the issue was not reproduced. All consumers were receiving good audio with no interruptions.
Tests conducted:
I. P on REMAP, C1 on WU, C2 on CSU 12:35¶
- C1 <-- P for 30sec John - OK
- C2 <-- P for 30sec, Jeff - OK
- C1 <-- P for 30sec + C2 <— P for 60-90sec, 30sec John - ok, +Jeff - OK
II. P on REMAP, C1 on WU, C2 on WU - 12:40¶
- C1 <-- P for 30sec John, no choppiness, audio OK 12:42
- C2 <-- P for 30sec Jeff, no choppiness
- C1 <-- P for 30sec + C2 <— P for 60-90sec, 30sec John - ok ,+Jeff - OK, no choppiness
III. P on REMAP, C1 on WU, C2 on UCLA 12:47¶
- C1 <-- P for 30sec, John no choppiness, audio OK
- C2 <-- P for 30sec, no choppiness, audio OK
- C1 <-- P for 30sec + C2 <— P for 60-90sec, 30sec John - ok, +Jeff - OK. no choppiness
IV. P on REMAP, C1 on WU, C2 on URJC 12:53¶
- C1 <-- P for 30sec, John, no choppiness, audio OK
- C2 <-- P for 30sec, Jeff, no choppinees, audio OK.
- C1 <-- P for 30sec + C2 <— P for 60-90sec, 30sec John - ok, +Jeff - OK
Dashboard with gathered metrics is here.
Suggestions for the next steps:
- run same tests with video stream (1Kbps)
- if issue does not reveal itself, increase number of consumers - 3, 4, 5, ...
- limit CPU resources on test machines and see how it affects performance
Updated by Peter Gusev over 8 years ago
- % Done changed from 60 to 80
Run similar tests for video. Results are here.
No major issues revealed yet. Will increase the number of consumers in future tests.
Minor issues:
Updated by Peter Gusev over 8 years ago
- Related to Task #3541: [ndnrtc] Increased latency after rebuffering added
Updated by Peter Gusev over 8 years ago
- File 1-at-WU-1-at-CSU.png 1-at-WU-1-at-CSU.png added
- File 2consumer-at-WU.png 2consumer-at-WU.png added
Attached screenshots of testbed traffic for video tests.
Updated by Peter Gusev about 8 years ago
Updated by Peter Gusev about 8 years ago
- Status changed from In Progress to Resolved