mGuard

# Repo

We currently use ndn-python-repo

![](repo.png)

## Development Summary: NDN Repo Integration & Performance Debugging in mGuard

### Background

The original mGuard implementation relied on the bulk-insert API from ndn-python-repo. Under realistic workloads, bulk insert proved unsuitable:

* the repo crashed when batches exceeded ~100 packets,

* even after fixes, insert operations began failing around ~600 packets,

* the bulk API provided no timing visibility, making debugging impossible.

To address this, a C++ Interest-based repo publisher was implemented to mirror the Python publisher’s behavior. The long-term objective is to turn this into a standalone reusable C++ module independent of mGuard.

---

### C++ Repo Publisher (Interest-Based Insertion)

The C++ publisher followed the notify → msg → data workflow defined by ndn-python-repo.

During integration, a key problem emerged:

at ~50 Hz data rates, the repo began timing out, not because of repo limitations, but because the producer was issuing insert operations faster than repo/NFD could process them.

This highlighted that the failure was not purely repo-related. NFD scheduling, face load, and filtering overhead all contributed to the observed timeouts.

To stabilize mGuard insert behavior, a QueueManager was added to the producer. It paced inserts and prevented bursts that previously caused repo crashes or inconsistent manifest publication.

---

### Repo Workflow vs. mGuard Workflow: Fundamental Mismatch

The repository’s architecture is optimized for the classical file model:

* one file name,

* multiple data segments,

* fetch pipeline: notify → msg → fetch many segments.

mGuard, however, produces the opposite workload:

* thousands of distinct names per stream,

* each name usually 1–2 segments (data + CK),

* every packet triggers its own full control-plane sequence.

As a result, the repo must execute:

* one notify per packet,

* one msg per packet,

* one data fetch per packet.

For a 5,000-packet batch:

* 5,000 notify Interests

* 5,000 msg Interests

* 5,000 fetch Interests

This creates extremely high Interest volume and rapidly stresses both repo and NFD, particularly when multiple interest filters and PSync traffic are also active.

This mismatch is the core reason mGuard stresses ndn-python-repo under high-rate streaming workloads.

---

### Key Findings

**1. Repo is not the primary bottleneck**

Experiments with an isolated publisher showed that repo alone can sustain serial insertions up to several thousand packets without failure. Failures in mGuard stem from repo load combined with:

* high Interest rate,

* many interest filters,

* PSync activity,

* NFD face congestion.

**2. Sustainable throughput is ~80–100 inserts/sec**

This is the practical ceiling after which notify/msg RTTs grow and timeouts appear.

mGuard should pace producer insert operations using a window of ≈ 10 packets to stay within this limit.

**3. Repo delays > 4 seconds cause hard failures**

If repo cannot fetch the actual data within 4 seconds after notify/msg exchange, insertion is aborted.

This tends to affect the first 1–2 packets in a batch, before the pipeline stabilizes.