ndn-sci

Road Map

========

Current Catalog Status

-----------------------

- We have an initial catalog implementation that supports menu (component type) and auto-complete NDN name queries against a MySQL database.

- Chengyu will implement name synchronization with chronosync.

- Susmit will implement the data provider's publishing module.

- All available CMIP5 names have been published into their respective databases.

Catalog Milestone Summary

--------------------------

The catalog implementation should be prioritize to present increasingly interesting concepts. We've started off simple: we can query a single catalog without specifying any particular host. This starting point can be evolved to demonstrate synchronization, data publication/retrieval, and security.

Security is currently a low priority milestone because we can demonstrate synchronization and data movement without it. Provenance is an important topic for scientists, but adds little on its own in terms of demo value (i.e. something is accepted or rejected). In the meantime, we can use an "anything goes" security policy.

We also need to fit query protocol improvements (e.g. union support) into the current milestones to improve user interactions with the catalog. However, the current focus is completing the initial catalog design (query, sync, and publication).

Similarly, the database schema, and the corresponding catalog SQL query functionality, may also need to be changed to avoid any scalability problems from using regex searches for autocomplete queries. However, the current schema seems to be good enough for now. There may be a more immediate problem from signing and verifying query results. In particular, queries asking for all cmip5 names will produce hundreds of packets. The NDN Javascript library will only be able to verify between 300 - 1,000 signatures/second, depending on the signing algorithm, once verification is enabled.

Catalog Milestones by Priority

-------------------------------

1. Synchronize catalogs using ChronoSync

  - Our last demo focused on querying and displaying results from a single catalog.

  - Allows us to start talking about catalogs as a system with a consistent view rather than independent hosts/databases.

  - Next demo could demonstrate NDN names being inserted from any catalog and the catalog system synchronizing.

2. Add dataset publication

3. Use manifests to improve dataset retrieval performance

  - Manifests are needed to avoid performing expensive public key signature verification.

  - Publishers will still need to use public key signatures on their data, unless we can justify moving to SHA256 digest "signatures".

3. Validate data publication and synchronization messages

  - Need to document trust model(s) and verification strategy for data publication.

4. Serve datasets directly from disk with a new NDN repo