Project

General

Profile

Task #3526

March 10 NDN-RTC test

Added by Peter Gusev about 3 years ago. Updated over 2 years ago.

Status:
Resolved
Priority:
Normal
Assignee:
Category:
-
Start date:
03/09/2016
Due date:
% Done:

80%

Estimated time:

Description

Tests on March 9th demonstrated poor performance in multi-consumer scenarios for audio conferencing.
There is a need to narrow down testing and understand issues causing poor performance (choppy audio).

2consumer-at-WU.png (1.5 MB) 2consumer-at-WU.png Peter Gusev, 03/10/2016 07:07 PM
1-at-WU-1-at-CSU.png (1.5 MB) 1-at-WU-1-at-CSU.png Peter Gusev, 03/10/2016 07:07 PM
Screen Shot 2016-03-09 at 2.46.03 PM (2).png (1.03 MB) Screen Shot 2016-03-09 at 2.46.03 PM (2).png Peter Gusev, 10/31/2016 04:01 PM
537
538
685

Related issues

Related to ndnrtc - Task #3541: [ndnrtc] Increased latency after rebufferingNew2016-03-10

History

#1 Updated by Peter Gusev about 3 years ago

The following test set was proposed:

  • topology:
    1 producer connected to REMAP, 2 consumers connected elsewhere on the testbed (among UCLA,CSU,WU,REMAP hubs)

  • general description:
    we’d like to explore how multiple consumers affect each other when fetching from the same producer. for that, one consumer will start fetching from producer, after some time (30sec) another consumer will start fetching from the same producer and we’ll see how this affects audio for both (does it affect at all). we will do several rounds of this test with one consumer connecting to different hubs on the testbed. before multi-consumer test, we’ll check that audio works decently for both consumers in 1-to-1 scenario.

  • detailed steps:

    • producer P publishes audio stream; connected to REMAP.
    • audio is checked for each consumer separately:
      • 1/ consumer C1 connects to WU and fetches audio for 30-60sec from P. confirm that audio works decently. consumer C1 stops fetching. report to chat.
      • 2/ consumer C2 connects to hub and fetches audio for 30-60sec from P. confirm that audio works decently. consumer C2 stops fetching. report to chat.
    • audio is checked for both consumers:
      • 3/ consumer C1 connects to hub and fetches audio from P. after 30sec, consumer connects to hub and starts fetching audio from P. observe audio quality for both consumers for 60-90sec. report to chat
  • tests set:

I. P on REMAP, C1 on WU, C2 on WU

  1. C1 <-- P for 30sec (C1 fetches form P for 30 sec and stops)
  2. C2 <-- P for 30sec (C2 fetches from P for 30 sec and stops)
  3. C1 <-- P for 30sec + C2 <— P for 60-90sec (C1 fetches from P for 30 sec, then C2 starts fetching form P for another 60-90sec, then both stop)

II. P on REMAP, C1 on WU, C2 on CSU

  1. C1 <-- P for 30sec (C1 fetches form P for 30 sec and stops)
  2. C2 <-- P for 30sec (C2 fetches from P for 30 sec and stops)
  3. C1 <-- P for 30sec + C2 <— P for 60-90sec (C1 fetches from P for 30 sec, then C2 starts fetching form P for another 60-90sec, then both stop)

III. P on REMAP, C1 on WU, C2 on UCLA

  1. C1 <-- P for 30sec (C1 fetches form P for 30 sec)
  2. C2 <-- P for 30sec (C2 fetches from P for 30 sec)
  3. C1 <-- P for 30sec + C2 <— P for 60-90sec (C1 fetches from P for 30 sec, then C2 starts fetching form P for another 60-90sec, then both stop)

IV. P on REMAP, C1 on WU, C2 on REMAP

  1. C1 <-- P for 30sec (C1 fetches form P for 30 sec)
  2. C2 <-- P for 30sec (C2 fetches from P for 30 sec)
  3. C1 <-- P for 30sec + C2 <— P for 60-90sec (C1 fetches from P for 30 sec, then C2 starts fetching form P for another 60-90sec, then both stop)

#2 Updated by Peter Gusev about 3 years ago

  • Status changed from New to In Progress
  • % Done changed from 0 to 60

Test results: the issue was not reproduced. All consumers were receiving good audio with no interruptions.

Tests conducted:

I. P on REMAP, C1 on WU, C2 on CSU 12:35

  1. C1 <-- P for 30sec John - OK
  2. C2 <-- P for 30sec, Jeff - OK
  3. C1 <-- P for 30sec + C2 <— P for 60-90sec, 30sec John - ok, +Jeff - OK

II. P on REMAP, C1 on WU, C2 on WU - 12:40

  1. C1 <-- P for 30sec John, no choppiness, audio OK 12:42
  2. C2 <-- P for 30sec Jeff, no choppiness
  3. C1 <-- P for 30sec + C2 <— P for 60-90sec, 30sec John - ok ,+Jeff - OK, no choppiness

III. P on REMAP, C1 on WU, C2 on UCLA 12:47

  1. C1 <-- P for 30sec, John no choppiness, audio OK
  2. C2 <-- P for 30sec, no choppiness, audio OK
  3. C1 <-- P for 30sec + C2 <— P for 60-90sec, 30sec John - ok, +Jeff - OK. no choppiness

IV. P on REMAP, C1 on WU, C2 on URJC 12:53

  1. C1 <-- P for 30sec, John, no choppiness, audio OK
  2. C2 <-- P for 30sec, Jeff, no choppinees, audio OK.
  3. C1 <-- P for 30sec + C2 <— P for 60-90sec, 30sec John - ok, +Jeff - OK

Dashboard with gathered metrics is here.

Suggestions for the next steps:

  • run same tests with video stream (1Kbps)
  • if issue does not reveal itself, increase number of consumers - 3, 4, 5, ...
  • limit CPU resources on test machines and see how it affects performance

#3 Updated by Peter Gusev about 3 years ago

  • % Done changed from 60 to 80

#4 Updated by Peter Gusev about 3 years ago

  • Related to Task #3541: [ndnrtc] Increased latency after rebuffering added

#5 Updated by Peter Gusev about 3 years ago

537
538

Attached screenshots of testbed traffic for video tests.

#7 Updated by Peter Gusev over 2 years ago

  • Status changed from In Progress to Resolved

Also available in: Atom PDF