Feature #3481
openTest NDN-RTC performance in Mini-NDN
Added by Jeff Burke over 8 years ago. Updated almost 7 years ago.
0%
Description
Create test harness and document so that others can try, as a way of evaluating application and forwarder performance under NDN-RTC load.
Updated by Jeff Burke over 8 years ago
- Assignee set to Ashlesh Gawande
@Peter, can you please coordinate with Ashlesh about this. Zhehao can also advise about using MiniNDN with real code.
Updated by Peter Gusev over 8 years ago
@Ashlesh, can you give a brief description on what is required to test C++ library in miniNDN?
Updated by Zhehao Wang over 8 years ago
Jeff Burke wrote:
@Peter, can you please coordinate with Ashlesh about this. Zhehao can also advise about using MiniNDN with real code.
A quick note on this: the way I was using it was deriving an Experiment from MiniNDN's set of experiments. Here's an example in a fork of MiniNDN.
Custom command is executed in each MiniNDN node (which runs its own NFD process). Topology is loaded in via command line (described in MiniNDN readme), and for my test, I established static routes based on a configuration file.
Assuming that the Ubuntu headless app works as intended, you should be able to do a similar thing to run your test binary that uses ndnrtc library.
One note though, if the headless app + NFD is going to be CPU heavy, you may not be able to run a large number of instances on one physical host.
Updated by Ashlesh Gawande over 8 years ago
I think Zhehao covered the basics.
We can run an instance of NDN-RTC on each node on Mini-NDN and run some tests. Are such tests already written?
If not how can I test NDN-RTC between two machines?
One can only test the headless version since Mini-NDN (Mininet, https://mailman.stanford.edu/pipermail/mininet-discuss/2014-May/004588.html) does not work on Mac.
Updated by Peter Gusev over 8 years ago
The headless app is under development right now.
What kind of tests are you expecting? Is there a specific description language for such tests or do you expect me to provide you configuration files for topologies?
Ideally, I would like to hear what do you need from me to run the headless app. I.e. headless app is invoked from command line with arguments, one of them is the NDN-RTC configuration. Will this functionality be sufficient?
Here's an edited excerpt from our earlier internal e-mail conversation about headless app general requirements:
Ubuntu NDN-RTC headless application.
Here are drafted requirements for Ubuntu headless app:
- ability to publish video streams video frames should be read from specified file on disk (check .yuv files) with configurable framerate
- ability to publish audio streams right now audio capturing is performed inside WebRTC which accesses default input device. as packet sizes do not depend on whether there’s actual sound or not (for the codec we currently use) we might leave it untouched. if we’ll see a need in sending actual audio, we might consider virtual audio devices as a fast remedy.
- ability to consume audio/video streams this is pretty straightforward. we also should provide a function of writing decoded video to a file (.yuv or argb) which can be later used by producer as well.
- configuration file there’s an example of configuration file (it’s parsed by libconfig++ library). I suggest using it and update if needed. invoking application should be as simple as this: $ ndnrtc-headless -c config.cfg
- statistics gathering ...query statistics and save them to a .stat files. I suggest to make selecting statistics configurable. I.e. somewhere in configuration file we provide a list of which statistics we want the app to track and a file name; and they are gathered in captioned tab-delimited files which will be easy to import in matlab/excel or similar. Should look something like that:
stat { buffer.stat: “buf est, buf tar, buf play”; playback.stat: “skipped no, incomplete, total”; ... }
This should give us enough flexibility to test gathering different sets of metrics to analyze streaming efficiency and troubleshoot faster.
Let me know, @Ashlesh, if this is helpful to get a grasp of what headless app will look like.
Also, I'm curios about few things in miniNDN in regards to headless app:
- audio/video publishing ability in miniNDN - what is the current situation with audio devices? are there any?
- write consumed raw video frames to files - 60 sec of 640x480 raw video may take around 1GB of disk memory and may become disk-write intensive. does miniNDN have any limitations in this regard? I might consider writing it into a compressed container or pipe it through something like ffmpeg eventually, but it'll require more dev time and initial solution will still be raw video frames.
Updated by Jeff Burke over 8 years ago
Dependent on completion of NDN-RTC headless producer/consumer (#3509).
Updated by Jeff Burke over 8 years ago
- Blocked by Task #3509: [ndnrtc-client] Headless client added