PSync Design¶

Introduction¶

PSync is used to sync stream names and their sequence numbers across applications on a network. Partial sync, where consumers and data generators are separate entities, uses separate “hello” and “sync” protocols to facilitate this synchronization.

Old PSync Implementation¶

Old Hello Protocol¶

The hello protocol is used to communicate the producer’s state (a list of streams with sequence numbers) in raw and encoded (IBF) forms. Consumers send Interests for/<sync-prefix>/hello . The response from the network is:
Name:

/<sync-prefix>/hello/<latest-producer-IBF>

Data:

/<stream-name>/<sequence-number>
/<stream-name>/<sequence-number>
...
/<stream-name>/<sequence-number>

The application can then choose which streams to add to their subscription list.

Old Sync Protocol¶

The “sync” protocol is then used to notify consumers when the producer has increased a sequence number for a stream that they are subscribed to. Consumers send Interests for /<sync-prefix>/sync/<SL>/<old-IBF>/ where <SL> is the list of streams the consumer wants to subscribe to. The producer will calculate the difference between <old-IBF> and its current IBF in the form of streams and sequence numbers.If there is a difference (consumer is out of sync with the producer) then the producer will respond with this Data:
Name

/<sync-prefix>/sync/<SL>/<old-IBF>/<new-IBF>

Data

/<stream-name>/<sequence-number>
/<stream-name>/<sequence-number>

Problems¶

Hello Data Names are not Unique
Hello data will change when a producer adds, updates, or removes a stream, but the data name does not change. Implementation details may be changed to prevent a consumer fetching old data, but it does not follow good design principles.

Consumers are not Notified of Changes to Available Streams
When the producer adds or removes a stream, consumers’ only mechanism for knowing of this update is through the Hello protocol. However, regularly sending Hello interests will render the sync protocol redundant, as sync data is always a subset of hello data.

• Hello requests used static names -> consumers got cached/outdated lists
• No automatic stream change notification
• User missed new data stream because it wasn't dynamically advertised
• Frequent polling defeated Sync's purpose

New PSync Implementation¶

To solve these problems, we propose integrating the hello protocol into the sync protocol as a data stream, named /<prefix>/DEFAULT , as the consumer is subscribed to it by default. The sync protocol has mechanisms for
• unique names for data streams
• notification of updates to data streams
• communication of producer IBF
which satisfy our needs.
Data for the default stream will consist of a list of available streams. Its data name will be/<prefix>/DEFAULT/<sequence-number> where the sequence number is increased by 1 any time a stream is added or removed. Updates to this sequence number will be synced with consumers via the sync protocol.

Summary of New Protocol¶

• Combining Hello Protocol and Sync Protocol
• Add a default stream: <producer-prefix>/DEFAULT/<seq>
• Consumers are always subscribed to it.
• Producer publishes stream updates (add/remove) to this stream.
• Default stream seq number updates after each change.
• Sync protocol delivers these updates like any other update.