mGuard

# PSync Design

## Introduction

PSync is used to sync stream names and their sequence numbers across applications on a network. Partial sync, where consumers and data generators are separate entities, uses separate **“hello”** and **“sync”** protocols to facilitate this synchronization.

---

## Old PSync Implementation

### Old Hello Protocol

The hello protocol is used to communicate the producer’s state (a list of streams with sequence numbers) in raw and encoded (IBF) forms. Consumers send Interests for``` /<sync-prefix>/hello``` . The response from the network is:

**Name:** 

```

/<sync-prefix>/hello/<latest-producer-IBF>

 ```

**Data:**

```

/<stream-name>/<sequence-number>

/<stream-name>/<sequence-number>

...

/<stream-name>/<sequence-number>

```

The application can then choose which streams to add to their subscription list.

### Old Sync Protocol

The **“sync”** protocol is then used to notify consumers when the producer has increased a sequence number for a stream that they are subscribed to. Consumers send Interests for ``` /<sync-prefix>/sync/<SL>/<old-IBF>/``` where ``` <SL>``` is the list of streams the consumer wants to subscribe to. The producer will calculate the difference between ```<old-IBF> ``` and its current IBF in the form of streams and sequence numbers.If there is a difference (consumer is out of sync with the producer) then the producer will respond with this Data:

**Name**

```

/<sync-prefix>/sync/<SL>/<old-IBF>/<new-IBF>

```

**Data**

```

/<stream-name>/<sequence-number>

/<stream-name>/<sequence-number>

```

### Problems

**Hello Data Names are not Unique**

Hello data will change when a producer adds, updates, or removes a stream, but the data name does not change. Implementation details may be changed to prevent a consumer fetching old data, but it does not follow good design principles.

**Consumers are not Notified of Changes to Available Streams**

When the producer adds or removes a stream, consumers’ only mechanism for knowing of this update is through the Hello protocol. However, regularly sending Hello interests will render the sync protocol redundant, as sync data is always a subset of hello data. 

* Hello requests used static names -> consumers got cached/outdated lists

* No automatic stream change notification

* User missed new data stream because it wasn't dynamically advertised

* Frequent polling  defeated Sync's purpose

---

## New PSync Implementation

To solve these problems, we propose integrating the hello protocol into the sync protocol as a data stream, named ```/<prefix>/DEFAULT``` , as the consumer is subscribed to it by default. The sync protocol has mechanisms for

• unique names for data streams

• notification of updates to data streams

• communication of producer IBF

which satisfy our needs.

Data for the default stream will consist of a list of available streams. Its data name will be``` /<prefix>/DEFAULT/<sequence-number>``` where the sequence number is increased by 1 any time a stream is added or removed. Updates to this sequence number will be synced with consumers via the sync protocol.

### Summary of New Protocol

* Combining Hello Protocol and Sync Protocol

* Add a default stream: ```<producer-prefix>/DEFAULT/<seq>```

* Consumers are always subscribed to it.

* Producer publishes stream updates (add/remove) to this stream.

* Default stream seq number updates after each change.

* Sync protocol delivers these updates like any other update.

![](general_sync.png)