Preliminary design description: https://docs.google.com/document/d/1eMwwIUu4DMmL4UxbuKSsZAeWkOpOM6cBygzBNH5IYcA/edit?usp=sharing

Where's the spec? If it's already on Gerrit, post the link here.

See the link in note 4

I will update the specs to reflect the new validator interface.

I've briefly read the spec.

It's unclear how "list of certificates" is encoded. This needs to be formally declared.

Must the KeyBundle contain a complete certificate chain? Are certificates required to appear in a certain order?

How does a Validator know whether the requester has prepared a KeyBundle or it should retrieve individual certificates instead?

Why does the KeyBundle carry the trust anchor certificate? Trust anchor is already known by the validator and should be omitted. In WebPKI, the web server does not send root CA certificate in its certificate chain.

I will add the packet spec on wiki here. I have updated the consumer side spec on the google doc.

The Bundle is expected to have the certificates in a certain order i.e. from the first certificate needed to validate the data packet itself, followed by the certificate needed to validate the first certificate, so on and so forth.

The validator first requests for a Certificate Bundle and if that interest times out or returns a Nack it will fetch individual certificates from the network.

Whether the bundle carries the trust anchor or not is undefined. The producer will create the bundle as per it's knowledge i.e. till it either reaches a self signed certificate or may be a loop or may be some const MAX_BUNDLE_SIZE.

Packet format specs : https://redmine.named-data.net/projects/ndn-cxx/wiki/Certificate_Bundle_Packet_Format

The Bundle is expected to have the certificates in a certain order i.e. from the first certificate needed to validate the data packet itself, followed by the certificate needed to validate the first certificate, so on and so forth.

This implies that a prover must have access to the trust schema in order to generate a bundle, because the trust schema defines which certificates are needed to verify a packet and in what order. This is a reasonable assumption.
A consequence is that the name of a bundle should indicate which trust schema is in use, because the same certificate may potentially be used by multiple applications with different trust schemas. Certificate_Bundle_Packet_Format rev1 includes <trust-model> component in a bundle name, but lacks an explanation on how the verifier can derive this name from the certificate name which is the only information available in KeyLocator.

Whether the bundle carries the trust anchor or not is undefined.

This needs more explanation in the documents, because it's unobvious for a reader who knows PKI fairly well, as WebPKI recommends not to include the trust anchor (root CA certificate) in the certificate chain sent by a TLS sever.
I do see a valid use case for including the trust anchor (even if it's self-signed) in the bundle: the verifier's trust schema may only contain a digest of the trust anchor certificate instead of the full certificate. In this case, including the trust anchor would allow the verifier to see the public key.

The producer will create the bundle as per it's knowledge i.e. till it either reaches a self signed certificate or may be a loop or may be some const MAX_BUNDLE_SIZE.

If the verifier encounters an incomplete bundle, i.e. it needs more certificates than what's available from the bundle, does the verifier attempt to retrieval additional certificates individually?

the certificates present in the bundle are cached in the unverified cache and the validation process continues on the first certificate present in the bundle. Note it is assumed that the certificate bundle has its certificates is in a certain order where the first certificate is the certificate needed to validate the signature of the target data packet.

If the certificate bundle only serves as a data source to populate the unverified certificate cache, what's the purpose of the recommendation that certificates are sorted in a certain order?

there is a version number because it’s possible that the Certificate Bundle is updated (in case some key in the chain is revoked)

How to ensure the verifier is retrieving the latest version, or at least a good enough version that does not contain a revoked/expired intermediate certificate, instead of an older version from a network cache that eventually causes the verificate to fail?

Each segment will always have complete certificates.

What's the benefit of this requirement?
The obvious drawback is not being able to use the full size of a Data packet, and impossible to encapsulate the certificate that is close to the size limit because the outer name needs extra bytes.
Note that "if the first segment contains enough certificates then we don't retrieve the second segment" is not a valid reason for requiring segment to stop at certificate boundary, because the stream constructed by contents from segments can be processed in the same way as NFD UnixStreamTransport reads LpPackets from a stream socket.

A consequence is that the name of a bundle should indicate which trust schema is in use, because the same certificate may potentially be used by multiple applications with different trust schemas. Certificate_Bundle_Packet_Format rev1 includes <trust-model> component in a bundle name, but lacks an explanation on how the verifier can derive this name from the certificate name which is the only information available in KeyLocator.

For now we only support the trust models with a single hierarchy. I will append a number "00" as a component in the Bundle name to represent that and add that in the specs too.

This needs more explanation in the documents, because it's unobvious for a reader who knows PKI fairly well, as WebPKI recommends not to include the trust anchor (root CA certificate) in the certificate chain sent by a TLS server.

I will add more explanation. Basically, if the trust anchor is agreed upon by both the producer and the consumer then we don't have to include it. But currently we don't know this. It's possible that a trust model as no fixed trust anchor. So it's undefined.

If the verifier encounters an incomplete bundle, i.e. it needs more certificates than what's available from the bundle, does the verifier attempt to retrieval additional certificates individually?

Yes.

If the certificate bundle only serves as a data source to populate the unverified certificate cache, what's the purpose of the recommendation that certificates are sorted in a certain order?

I would say it's better to keep it sorted because we don't want to fetch all the segments if we don't have to. So if the certificates are in random order in the bundle it might lead to some additional fetches which could have been avoided if the certificates were in order.

How to ensure the verifier is retrieving the latest version, or at least a good enough version that does not contain a revoked/expired intermediate certificate, instead of an older version from a network cache that eventually causes the verification to fail?

Currently, we only rely on freshness period and must be fresh.

What's the benefit of this requirement?

Actually, this is not a hard requirement. May be I can mention that in the spec that it is mostly for simplicity.

For now we only support the trust models with a single hierarchy. I will append a number "00" as a component in the Bundle name to represent that and add that in the specs too.

Are you saying that each data-signing-certificate is usable with one trust schema? This is too much restriction and is unacceptable.

It's possible that a trust model as no fixed trust anchor.

WebPKI has many root CAs, but the issuer of the highest level of intermediate CA is the trust anchor being used. Similarly, an NDN trust schema can have multiple trust anchors, and the bundle only needs to include up to the certificate directly issued by one of the trust anchors, but not the self-signed trust anchor. An exception is the verifier does not have the public key of the trust anchor, as described in note-17.

If the certificate bundle only serves as a data source to populate the unverified certificate cache, what's the purpose of the recommendation that certificates are sorted in a certain order?

I would say it's better to keep it sorted because we don't want to fetch all the segments if we don't have to. So if the certificates are in random order in the bundle it might lead to some additional fetches which could have been avoided if the certificates were in order.

Agreed. This should be a SHOULD-level requirement rather than a MUST-level, and the rationale needs to explained in the document.

How to ensure the verifier is retrieving the latest version, or at least a good enough version that does not contain a revoked/expired intermediate certificate, instead of an older version from a network cache that eventually causes the verification to fail?

Currently, we only rely on freshness period and must be fresh.

As you know, these are mostly useless. This should appear as a known limitation in the documents.

Each segment will always have complete certificates.
What's the benefit of this requirement?

Actually, this is not a hard requirement. May be I can mention that in the spec that it is mostly for simplicity.

Are you saying this is a MAY-level requirement? If so, verifier needs to be able to accept a bundle in which segment does not stop at certificate boundary, since a MAY-level requirement is truly optional at prover end.

Junxiao Shi wrote:

For now we only support the trust models with a single hierarchy. I will append a number "00" as a component in the Bundle name to represent that and add that in the specs too.

Are you saying that each data-signing-certificate is usable with one trust schema? This is too much restriction and is unacceptable.

I don't understand this comment. What Manika mentioned is that the current bundle can help with certificate retrieval for any trust schema that uses a single chain of certificates to verify a data packet (this is what single hierarchy is meant, it is not related in any way to name hierarchy or any restrictions on data/cert names). In principle, bundle can help Web-of-Trust models, but this will be defined/implemented, probably specific to an applications.

It's possible that a trust model as no fixed trust anchor.

WebPKI has many root CAs, but the issuer of the highest level of intermediate CA is the trust anchor being used. Similarly, an NDN trust schema can have multiple trust anchors, and the bundle only needs to include up to the certificate directly issued by one of the trust anchors, but not the self-signed trust anchor. An exception is the verifier does not have the public key of the trust anchor, as described in note-17.

Bundle doesn't by itself impose trust model. It is just a combined set of certificates. It doesn't have knowledge of specific trust anchors, so whatever is included is included. Ultimately, it is Validator task to get proper certificates. Carrying them in a bundle is just a helper.

Each segment will always have complete certificates.
What's the benefit of this requirement?

Actually, this is not a hard requirement. May be I can mention that in the spec that it is mostly for simplicity.

Are you saying this is a MAY-level requirement? If so, verifier needs to be able to accept a bundle in which segment does not stop at certificate boundary, since a MAY-level requirement is truly optional at prover end.

As of now, it is MUST-level requirement. The goal for now to make implementation simple. This may change in the future and we may change the validator implementation, but I do not want to overcomplicate things for now.

the current bundle can help with certificate retrieval for any trust schema that uses a single chain of certificates to verify a data packet

An example in WebPKI: Let’s Encrypt Authority X3 intermediate is issued by ISRG Root X1 and cross-signed by IdenTrust root. Before ISRG Root X1 is trusted by browsers, certificate chains will look like "example.com - Let’s Encrypt Authority X3 (issuer=IdenTrust)". After ISRG Root X1 becomes trusted by browsers, certificate chains will look like "example.com - Let’s Encrypt Authority X3 (issuer=ISRG)". Note that Let’s Encrypt Authority X3 (issuer=IdenTrust) and Let’s Encrypt Authority X3 (issuer=ISRG) are two different certificates but they contain same public key.

Bringing this example to NDN trust schema: seeing example.com's certificate, the trust schema installed in Firefox 49 or below will derive Let’s Encrypt Authority X3 (issuer=IdenTrust) as the issuer, while the trust schema installed in Firefox 50 or above will derive Let’s Encrypt Authority X3 (issuer=ISRG) as the issuer.
How does the TLS server which intermediate should be sent in the bundle, if the bundle name does not contain an indicate on which trust schema is in use?

On the other hand, if we eliminate the requirement of sorting the certificates, this problem can be solved by including both Let’s Encrypt Authority X3 (issuer=IdenTrust) and Let’s Encrypt Authority X3 (issuer=ISRG) in the bundle, and verifier will take what they need from the unverified cache.

bundle can help Web-of-Trust models, but this will be defined/implemented, probably specific to an applications.

In Web-of-Trust, there are multiple issuers for one certificate and all those issuers' certificates must be retrieved and verified. I don't think NDN trust schema can support such model, and thus it's not a problem specific to bundle.

Bundle doesn't by itself impose trust model. It is just a combined set of certificates. It doesn't have knowledge of specific trust anchors, so whatever is included is included.

What would be the benefit of including a self-signed certificate (regardless of whether it is a trust anchor)? other than the situation in note-17.

As of now, it is MUST-level requirement. The goal for now to make implementation simple. This may change in the future and we may change the validator implementation, but I do not want to overcomplicate things for now.

As indicated in note-17, there is an obvious drawback, and the so-called simplification is invalid because exactly same procedure is already implemented in UnixStreamTransport and can be abstracted into a reusable library routine.
A real simplification is: just use SegmentFetcher and retrieve all segments, and leave "stop retrieving after getting enough segments" for later optimization.

Junxiao Shi wrote:

the current bundle can help with certificate retrieval for any trust schema that uses a single chain of certificates to verify a data packet

An example in WebPKI: Let’s Encrypt Authority X3 intermediate is issued by ISRG Root X1 and cross-signed by IdenTrust root. Before ISRG Root X1 is trusted by browsers, certificate chains will look like "example.com - Let’s Encrypt Authority X3 (issuer=IdenTrust)". After ISRG Root X1 becomes trusted by browsers, certificate chains will look like "example.com - Let’s Encrypt Authority X3 (issuer=ISRG)". Note that Let’s Encrypt Authority X3 (issuer=IdenTrust) and Let’s Encrypt Authority X3 (issuer=ISRG) are two different certificates but they contain same public key.

Bringing this example to NDN trust schema: seeing example.com's certificate, the trust schema installed in Firefox 49 or below will derive Let’s Encrypt Authority X3 (issuer=IdenTrust) as the issuer, while the trust schema installed in Firefox 50 or above will derive Let’s Encrypt Authority X3 (issuer=ISRG) as the issuer.
How does the TLS server which intermediate should be sent in the bundle, if the bundle name does not contain an indicate on which trust schema is in use?

On the other hand, if we eliminate the requirement of sorting the certificates, this problem can be solved by including both Let’s Encrypt Authority X3 (issuer=IdenTrust) and Let’s Encrypt Authority X3 (issuer=ISRG) in the bundle, and verifier will take what they need from the unverified cache.

Why you bringing TLS here? You just effectively described a form of web-of-trust (multi-cert) model. This is not supported in our initial implementation of bundle fetcher nor bundle helper, but is irrelevant from the spec point of view. Bundle includes all certificates potentially needed, some may not be actually used. If use case of cert bundle will get to the point of specific trust model with specific trust anchors, then one free to construct a non-redundant bundle, but I don't know exactly where you going with it.

bundle can help Web-of-Trust models, but this will be defined/implemented, probably specific to an applications.

In Web-of-Trust, there are multiple issuers for one certificate and all those issuers' certificates must be retrieved and verified. I don't think NDN trust schema can support such model, and thus it's not a problem specific to bundle.

Trust schema can support anything. You're referring a challenge in certificate retrieval (signature discovery). This is totally different question and outside the scope for both current schema and cert bundle specs. We have been discussing a few proposals about this and there was one implementation (in ChronoChat) for that, but nothing is definitive yet.

Bundle doesn't by itself impose trust model. It is just a combined set of certificates. It doesn't have knowledge of specific trust anchors, so whatever is included is included.

What would be the benefit of including a self-signed certificate (regardless of whether it is a trust anchor)? other than the situation in note-17.

Ok. This is a different conversation. Self-signed cert != trust anchor both ways. Self-signed cert can be a trust anchor or not be a trust anchor. And trust anchor can be self-signed or can be a non-self signed.

To answer your latest questions. Yes, I don't immediately see a value of including self-signed cert regardless its role in trust schema.

As of now, it is MUST-level requirement. The goal for now to make implementation simple. This may change in the future and we may change the validator implementation, but I do not want to overcomplicate things for now.

As indicated in note-17, there is an obvious drawback, and the so-called simplification is invalid because exactly same procedure is already implemented in UnixStreamTransport and can be abstracted into a reusable library routine.
A real simplification is: just use SegmentFetcher and retrieve all segments, and leave "stop retrieving after getting enough segments" for later optimization.

No. The point is to fetch only certs (bundle parts) that are actually needed. For first data packet under namespace, this could be all segments of the bundle, for others (in sub-branches), only one or two may be needed. That's the whole idea of not using the SegmentFetcher.

You just effectively described a form of web-of-trust (multi-cert) model. This is not supported in our initial implementation of bundle fetcher nor bundle helper

Isn't it already supported, since the prover can put into the bundle more certificates than needed by trust schema?

No. The point is to fetch only certs (bundle parts) that are actually needed. For first data packet under namespace, this could be all segments of the bundle, for others (in sub-branches), only one or two may be needed. That's the whole idea of not using the SegmentFetcher.

Yes, I figured that, but this is an optimization, not a requirement.
On the other hand, being able to include a certificate whose size is very close to Data packet size limit is a requirement for correct protocol operation.

Junxiao Shi wrote:

You just effectively described a form of web-of-trust (multi-cert) model. This is not supported in our initial implementation of bundle fetcher nor bundle helper

Isn't it already supported, since the prover can put into the bundle more certificates than needed by trust schema?

Technically yes, it is just not a current focus. That's why I said not supported.

ContentType MUST be omitted.

Why?

The producer MAY indicate the number of segments in a certificate bundle [...] using a SegmentNameComponent reflecting the last segment number

Why not a FinalBlockId?

ContentType MUST be omitted.

Why?

The name alone identifies the content type. There's no need for additional identification.
Allowing "blob" type doesn't convey additional meaning, but only wastes bytes.

The producer MAY indicate the number of segments in a certificate bundle [...] using a SegmentNameComponent reflecting the last segment number

Why not a FinalBlockId?

The bundle spec is defined to adopt Naming Conventions rev2, so there's no need for wrapping into FinalBlockId.

Junxiao Shi wrote:

ContentType MUST be omitted.

Why?

The name alone identifies the content type. There's no need for additional identification.
Allowing "blob" type doesn't convey additional meaning, but only wastes bytes.

32=cert-bundle is 13 bytes, certificates are hundreds of bytes each... and you're worried about 3 bytes?

The producer MAY indicate the number of segments in a certificate bundle [...] using a SegmentNameComponent reflecting the last segment number

Why not a FinalBlockId?

The bundle spec is defined to adopt Naming Conventions rev2, so there's no need for wrapping into FinalBlockId.

Still, FinalBlockId sounds more general to me.

ContentType MUST be omitted.

Why?

The name alone identifies the content type. There's no need for additional identification.
Allowing "blob" type doesn't convey additional meaning, but only wastes bytes.

32=cert-bundle is 13 bytes, certificates are hundreds of bytes each... and you're worried about 3 bytes?

Not just that. Disallowing ContentType improves the uniqueness of encodings.
Yes I know non-uniqueness could come from many other places, but it's better to reduce such places than to introduce them.
And no, other older protocols that allow such ambiguity are not reasons for this protocol to make the same mistake.

The producer MAY indicate the number of segments in a certificate bundle [...] using a SegmentNameComponent reflecting the last segment number

Why not a FinalBlockId?

The bundle spec is defined to adopt Naming Conventions rev2, so there's no need for wrapping into FinalBlockId.

Still, FinalBlockId sounds more general to me.

No, this is an application layer protocol and there's no reason to copy from network layer.