Project

General

Profile

Repo Protocol Specification » History » Version 59

Shuo Chen, 02/24/2014 01:45 PM

1 1 Shuo Chen
Repo Protocol Specification
2
===========================
3
4 36 Shuo Chen
A Repo supports the network by preserving content and responding to Interests requesting content that it holds. A Repo can exist in any node, and is recommended if applications in that node need to preserve data. The NDN repo protocol is a specification of repo operations including reading, insertion and deletion of data objects in repo.
5 2 Shuo Chen
6
Repo semantics is based on [Command Interests](http://redmine.named-data.net/projects/nfd/wiki/Command_Interests) with signed commponents an the end of the name and basic common semantics of NDN. 
7
8 11 Shuo Chen
When some operation of repo including insertion and deletion of data objects is requested, a command interest is sent. The command interest is the interest as command of insertion and deletion and signed as form of command interest for access control. The repo will response the command with the data object.
9 4 Shuo Chen
10
The repo protocol can be categorized in three sections of reading, insertion and deletion of data objects.
11
12
## Repo Reading Specification
13
14 36 Shuo Chen
Repo registers prefixes of data objects it holds into NDN fowarding deamon and the repo will respond the data with such prefixes.
15 6 Shuo Chen
16 11 Shuo Chen
A standard interest is used to fetch content from the repo. The repo will respond when the name of the interest matches the prefix it registered in NFD. If the content in repo matches the interests, it will respond with the data object. When the interest is not matched, it will not respond.
17 6 Shuo Chen
18
The protocol is listed as bellow
19
20
If there is a matching data object:
21
22 7 Shuo Chen
    Requester                     Repo
23
        |                           |
24
        |                           |
25
        |         Interest          |
26
     t1 |-------------------------->|
27
        |                           |
28
        |        Data Object        |
29
     t2 |<==========================|
30 1 Shuo Chen
        |                           |
31 7 Shuo Chen
        |                           |
32 1 Shuo Chen
        |                           |
33 7 Shuo Chen
34
If there is no matching data object:
35 6 Shuo Chen
36
    Requester                     Repo
37
        |                           |
38 1 Shuo Chen
        |                           |
39 7 Shuo Chen
        |         Interest          |
40
     t1 |-------------------------->|
41
        |                           |
42
        |                           |
43
        |                           |
44
45 30 Shuo Chen
## Repo Command
46
For insertion, deletion and other operations of repo, these commands are encoded in the form of [sigend command interest](http://redmine.named-data.net/projects/nfd/wiki/Command_Interests). The semantics of repo command interest is as follows:
47 14 Shuo Chen
48 8 Shuo Chen
The name semantics is defined to have following components:
49 1 Shuo Chen
50 17 Shuo Chen
* ``<name of repo>`` refers to specific name of repo
51 30 Shuo Chen
* ``<command verb>`` refers to the name of command
52
* ``<RepoCommandParameters>`` refers to parameters of repo command
53 1 Shuo Chen
54 38 Shuo Chen
The following components are components of singed interest for access control:
55 1 Shuo Chen
56
* ``<timestamp>``
57 8 Shuo Chen
* ``<random-value>``
58
* ``<SignatureInfo>``
59
* ``<SignatureValue>``
60
61 30 Shuo Chen
For name of the repo /ucla/cs/repo/, the command will be defined as this:
62 33 Shuo Chen
63 30 Shuo Chen
    /ucla/cs/repo/<command verb>/<RepoCommandParameters>/<timestamp>/<random-value>/<SignatureInfo>/<SignatureValue>
64 14 Shuo Chen
65 30 Shuo Chen
### RepoCommandParameters
66 29 Shuo Chen
67 48 Shuo Chen
    RepoCommandParameters ::= REPOCOMMANDPARAMETERS-TYPE TLV-LENGTH
68 1 Shuo Chen
                               Name?
69 42 Shuo Chen
                               Selectors?
70 1 Shuo Chen
                               StartBlockId?
71
                               EndBlockId?
72 29 Shuo Chen
73 48 Shuo Chen
    Name                  ::= NAME-TYPE TLV-LENGTH NameComponent*
74
    NameComponent         ::= NAME-COMPONENT-TYPE TLV-LENGTH BYTE+
75 43 Shuo Chen
76
    Selectors             ::= SELECTORS-TYPE TLV-LENGTH
77
                               MinSuffixComponents?
78
                               MaxSuffixComponents?
79
                               PublisherPublicKeyLocator?
80
                               Exclude?
81 47 Shuo Chen
                               ChildSelector?
82 43 Shuo Chen
83
    MinSuffixComponents   ::= MIN-SUFFIX-COMPONENTS-TYPE TLV-LENGTH
84
                               nonNegativeInteger
85
86
    MaxSuffixComponents   ::= MAX-SUFFIX-COMPONENTS-TYPE TLV-LENGTH
87
                               nonNegativeInteger
88
89
    PublisherPublicKeyLocator ::= KeyLocator
90
91
    Exclude               ::= EXCLUDE-TYPE TLV-LENGTH Any? (NameComponent (Any)?)+
92
    Any                   ::= ANY-TYPE TLV-LENGTH(=0)
93 47 Shuo Chen
94
    ChildSelector         ::= CHILD-SELECTOR-TYPE TLV-LENGTH
95
                               nonNegativeInteger
96 1 Shuo Chen
    
97
    StartBlockId          ::= STARTBLOCKID-TYPE TLV-LENGTH
98 49 Shuo Chen
                               nonNegativeInteger
99 1 Shuo Chen
    
100
    EndBlockId            ::= ENDBLOCKID-TYPE TLV-LENGTH
101 49 Shuo Chen
                               nonNegativeInteger
102 1 Shuo Chen
103 45 Shuo Chen
#### Name
104
105 50 Shuo Chen
Name in command parameter is TLV-encoded block and represents the name or prefix of the data repo will process. The definition and format of name section are the same as the [name section](http://named-data.net/doc/ndn-tlv/name.html#name) of NDN interest.
106 45 Shuo Chen
107 44 Shuo Chen
#### Repo Command Selectors
108 38 Shuo Chen
109 52 Shuo Chen
Repo command supports parts of interest selectors of interest to indicate which contents to process. The definition of standard NDN selectors is described in [NDN Selectors Doc](http://named-data.net/doc/ndn-tlv/interest.html#selectors). The concrete definitions of both standard NDN selectors and repo command selectors are the same.
110
111
The difference between standard NDN interest and Repo Deletion Command interest that, the standard NDN selectors just matches **one** data packet that conforms to the selector conditions, but repo command selectors would matches **any** data packets. For example, if Interest is expressed for /ndn/edu and Exclude specifies one name component ucla, in standard NDN interest, the data producers will first exclude all the data packets with prefix /ndn/edu but not /ndn/edu/ucla, and then just selects one data packet. In repo deletion command interest, it will select all the data packets with prefix /ndn/edu but not /ndn/edu/ucla. However, in repo insert command, the repo will fetch one data just like standard interest selectors.
112 38 Shuo Chen
113 51 Shuo Chen
Repo command supports parts of standard NDN interests including MinSuffixComponents, MaxSuffixComponents, PublisherPublicKeyLocator, Exclude, ChildSelector. If command contains other selectors, repo will ignore these not supported selectors. In addition, selectors are just supported in delete command. If other commands contains selectors, repo will ignore selectors when processing these commands.
114 38 Shuo Chen
115
The form of selectors is as follows:
116
117
    Selectors             ::= SELECTORS-TYPE TLV-LENGTH
118
                               MinSuffixComponents?
119
                               MaxSuffixComponents?
120
                               PublisherPublicKeyLocator?
121 1 Shuo Chen
                               Exclude?
122 51 Shuo Chen
                               ChildSelector?
123 38 Shuo Chen
    
124
    MinSuffixComponents   ::= MIN-SUFFIX-COMPONENTS-TYPE TLV-LENGTH
125
                               nonNegativeInteger
126
127
    MaxSuffixComponents   ::= MAX-SUFFIX-COMPONENTS-TYPE TLV-LENGTH
128
                               nonNegativeInteger
129
130
    PublisherPublicKeyLocator ::= KeyLocator
131
    
132
    Exclude               ::= EXCLUDE-TYPE TLV-LENGTH Any? (NameComponent (Any)?)+
133 1 Shuo Chen
    Any                   ::= ANY-TYPE TLV-LENGTH(=0)
134 51 Shuo Chen
135
    ChildSelector         ::= CHILD-SELECTOR-TYPE TLV-LENGTH
136
                               nonNegativeInteger
137 46 Shuo Chen
138
#### StartBlockId, EndBlockId
139 38 Shuo Chen
140 54 Shuo Chen
StartBlockId and EndBlockId are used to process segmented data. StartBlockId indicate the first segment number and EndBlockId indicate the last segment number. Repo will process segment data whose segment id between StartBlockId and EndBlockId. If StartBlockId is missing, the first segment id the repo process is 0; If EndBlockId is missing, this scenario is described in specific process in Repo Insertion Command section and Repo Deletion Command section.
141 53 Shuo Chen
142 30 Shuo Chen
## Repo Command Response
143 1 Shuo Chen
144 30 Shuo Chen
Repo command response is the response data packet of repo command interest. The response contains statuscode to indicate the status of command process and other information. A TLV-encoded block called ``RepoCommandResponse`` is encoded in content of the data packet.
145 1 Shuo Chen
146 30 Shuo Chen
    RepoCommandResponse   ::= INSERTSTATUS-TYPE TLV-LENGTH
147 55 Shuo Chen
                               ResponseId?
148 1 Shuo Chen
                               StatusCode?
149
                               StartBlockId?
150
                               EndBlockId?
151
                               InsertNum?
152 30 Shuo Chen
                               DeleteNum?
153 1 Shuo Chen
154 55 Shuo Chen
    ResponseId            ::= RESPONSEID-TYPE TLV-LENGTH
155
                                nonNegativeInteger 
156
    
157 1 Shuo Chen
    StatusCode            ::= STATUSCODE-TYPE TLV-LENGTH
158
                                nonNegativeInteger    
159
160
    StartBlockId          ::= STARTBLOCKID-TYPE TLV-LENGTH
161
                                nonNegativeInteger
162
    
163
    EndBlockId            ::= ENDBLOCKID-TYPE TLV-LENGTH
164
                                nonNegativeInteger
165 17 Shuo Chen
166
    InsertNum             ::= INSERTNUM-TYPE TLV-LENGTH
167
                                nonNegativeInteger
168
169 30 Shuo Chen
    DeleteNum             ::= DELETENUM-TYPE TLV-LENGTH
170
                                nonNegativeInteger
171 17 Shuo Chen
172 59 Shuo Chen
### ResponseId
173
174
### StatusCode
175
176
### StartBlockId, EndBlockId
177
178
### InsertNum, DeleteNum
179
180
181 58 Shuo Chen
## Repo Trust Model
182 57 Shuo Chen
183
The trust model of repo depends on people who deploy the repo service, such as PKI. Repo can specify their own verification policies, and data consumers can specify their own trust anchors. The NDN [FAQ](http://named-data.net/project/faq/#How_does_NDN8217s_8220trust_management8221_work) shows how NDN trust managment works.
184
185 30 Shuo Chen
## Repo Insertion Specification
186
187
Insertion specification is for some client and application to insert data objects into certain NDN repo. A command interest as insertion command will be sent to the repo. This command interest is a signed interest and will be validated with access control policy defined by the repo. When the interested is validated and name of the data is not existed in the repo. The repository will reponse with a data object containing OK status and start to send the interest to fetch the data to insert.
188
189
Segmented data insertion is also supported in the insertion protocol. Segmantation info is defined as a TLV-encoded SegmantationInfo block. If the content is segmented, the final segment id will be encoded in this block.
190
191
### Insertion command interest semantics
192
193
The name semantics is follows the format of the repo command. The <command verb> is defined as ``insert``.
194
For example, for ``<name of repo>`` as ``/ucla/cs/repo``, the following is an example:
195 1 Shuo Chen
196 31 Shuo Chen
    /ucla/cs/repo/insert/<RepoCommandParameters>/<timestamp>/<random-value>/<SignatureInfo>/<SignatureValue>
197 30 Shuo Chen
198
### Insertion status check
199
200
During the insertion progress, the requester could send insertion status check command to check the status of the insertion progress. This status check command is also signed interest. The semantics of insertion status check command is shown as follows:
201
202
<command verb> equals to ``insert check``. For example:
203 1 Shuo Chen
204 32 Shuo Chen
    /ucla/cs/repo/insert check/<RepoCommandParameters>/<timestamp>/<random-value>/<SignatureInfo>/<SignatureValue>
205 30 Shuo Chen
206 40 Shuo Chen
### Insert and Insert check command seletors
207
208
Selectors are not supported in these commands. Repo will ignore selectors when processing these commands.
209
210 30 Shuo Chen
### Insertatation status response
211
212
This insert status data object could be the response data object of both insert command and insert check command. It will follow the format of repo command response.
213
214
StatusCode indicates the status of insertion. InsertNum represents how many data has been intersted into the repo. StartBlockId and EndBlockId is the start and end segment id of data inserted. InsertNum is the how many data segments have been inserted.
215
216
For insert cammand, status code will be set according to definition below, StartBlockId and EndBlockId will be set according to the insert command.
217
218
For insert check command, status code will be set according to definition below, StartBlockId and EndBlockId will be set according to the insert command, insertnum will be set according to insertion progress.
219
220 1 Shuo Chen
StatusCode Definition:
221 17 Shuo Chen
222
``StatusCode`` | Description
223
-------------- | ----------------------------------------------
224 19 Shuo Chen
``100``        | The command is OK. can start to fetch the data
225
``200``        | All the data has been inserted
226 17 Shuo Chen
``300``        | This insertion is in progress
227
``401``        | This insertion check is invalidated
228
``404``        | No such this insertion is in progress
229 30 Shuo Chen
``505``        | status check command is not supported
230 17 Shuo Chen
231 22 Alex Afanasyev
### Protocol Process
232
233 20 Alex Afanasyev
1. start to authorize the command; if authorization does not fail immediately, go to step 3
234 17 Shuo Chen
235 20 Alex Afanasyev
2. send a negative response indicating authorization failure, and abort these steps
236 14 Shuo Chen
237 20 Alex Afanasyev
3. if both StartBlockId and EndBlockId are missing, go to step 6
238 14 Shuo Chen
239 20 Alex Afanasyev
4. if both StartBlockId and EndBlockId are present, and StartBlockId is less than or equal to EndBlockId, go to step 6
240 14 Shuo Chen
241 20 Alex Afanasyev
5. send a negative response indicating malformed command, and abort these steps
242 14 Shuo Chen
243 20 Alex Afanasyev
6. wait for authorization completion
244 14 Shuo Chen
245 20 Alex Afanasyev
7. if authorization fails, go to step 2
246 14 Shuo Chen
247 20 Alex Afanasyev
8. send a positive response indicating insert is in progress
248 14 Shuo Chen
249 20 Alex Afanasyev
9. if both StartBlockId and EndBlockId are present, go to step 15
250 14 Shuo Chen
251 20 Alex Afanasyev
10. start to retrieve Name
252 14 Shuo Chen
253 20 Alex Afanasyev
11. wait for retrieval completion
254 14 Shuo Chen
255 20 Alex Afanasyev
12. if retrieval fails, abort these steps
256 14 Shuo Chen
257 20 Alex Afanasyev
13. store retrieved Data packet
258 14 Shuo Chen
259 20 Alex Afanasyev
14. abort these steps
260 14 Shuo Chen
261 20 Alex Afanasyev
15. append StartBlockId to Name
262 1 Shuo Chen
263 20 Alex Afanasyev
16. start to retrieve Name
264 14 Shuo Chen
265 20 Alex Afanasyev
17. wait for retrieval completion
266 14 Shuo Chen
267 20 Alex Afanasyev
18. if retrieval fails, abort these steps
268 14 Shuo Chen
269 20 Alex Afanasyev
19. store retrieved Data packet
270 15 Shuo Chen
271 20 Alex Afanasyev
20. if retrieved Data packet contains FinalBlockId, and FinalBlockId is less than EndBlockId, let EndBlockId be FinalBlockId
272 15 Shuo Chen
273 20 Alex Afanasyev
21. if the last component of Name is greater than or equal to EndBlockId, abort these steps
274 15 Shuo Chen
275 20 Alex Afanasyev
22. increment the last component of Name
276 15 Shuo Chen
277 20 Alex Afanasyev
23. go to step 16
278 15 Shuo Chen
279 20 Alex Afanasyev
Implementation MAY pipeline the Interests.
280 1 Shuo Chen
281 20 Alex Afanasyev
### Repo command progress report
282 16 Shuo Chen
283 20 Alex Afanasyev
Implementation MAY publish a notification of status regarding insert progress. The process of status check is as follows:
284 18 Shuo Chen
285 20 Alex Afanasyev
1. start to authorize the insert status command
286 1 Shuo Chen
287 20 Alex Afanasyev
2. send a negative response indicating authorization failure, and abort these steps
288 18 Shuo Chen
289 20 Alex Afanasyev
3. start to check the progress of the insert with the data name in the command. If no such progress is found, go to 4. or go to 5.
290 18 Shuo Chen
291 20 Alex Afanasyev
4. reponse status with status code of 404
292 18 Shuo Chen
293 20 Alex Afanasyev
5. ckeck te status of insertion. return the status data content
294
295 17 Shuo Chen
### Protocol diagram:
296 10 Alex Afanasyev
297
    Requester                     Repo                          Data producer
298
        |                           |                                 |
299
        |                           |                                 |
300
      +---+  Insert command       +---+                               |
301
      |   | --------------------> |   |                               |
302
      +---+                       |   |                               |
303
        |                         |   |                               |
304
      +---+   Confirm start       |   |                               |
305
      |   | <==================== |   |                               |
306
      +---+   Reject command      +---+                               |
307
        |     (with status code)    |                                 |
308
        |                         +---+     Interest for Data       +---+
309 1 Shuo Chen
        |                         |   | --------------------------> |   |
310
        |                         +---+                             |   |
311 10 Alex Afanasyev
        |                           |                               |   |
312
        |                         +---+       Data segment          |   |
313
        |                         |   | <========================== |   |
314
        |                         +---+                             +---+
315
        |                           |                                 |
316
        |                           ~                                 ~
317
        |                           ~                                 ~
318
        |                           |                                 |
319
        |                         +---+     Interest for Data       +---+
320
        |                         |   | --------------------------> |   |
321
        |                         +---+                             |   |
322 1 Shuo Chen
        |                           |                               |   |
323
        |                         +---+       Data segment          |   |
324
        |                         |   | <========================== |   |
325
        |                         +---+                             +---+
326
        |                           |                                 |
327
        |                           |                                 |
328 17 Shuo Chen
        |                           ~                                 ~
329 1 Shuo Chen
        |                           ~                                 ~
330
        |                           |                                 |
331 30 Shuo Chen
        |                           |                                 |
332 19 Shuo Chen
        |                           |                                 |
333 30 Shuo Chen
      +---+   Status interest     +---+                               |
334 19 Shuo Chen
      |   | --------------------> |   |                               |
335
      +---+                       |   |                               |
336
        |                         |   |                               |
337
      +---+    Status response    |   |                               |
338
      |   | <==================== |   |                               |
339
      +---+                       +---+                               |
340
        |                           |                                 |
341
        |                           |                                 |
342
343
## Repo Deletion Specification
344
345
Deletion of one content object or content objects under certain prefix are both supported in repo specification. Selectors are used to select multiple content objects. These selectors are different from conventional [selectors](http://named-data.net/doc/ndn-tlv/interest.html#selectors) of interest packet. The conventional selecors of interest would select one data packet of repo. Selecors of delete command will return any data packets which applies for these selectors.
346
347
### Deletion command interest semantics
348
349 1 Shuo Chen
Deletion command interest follows the format of the repo command. For example:
350 19 Shuo Chen
351
    /ucla/cs/repo/delete/<RepoCommandParameters>/<timestamp>/<random-value>/<SignatureInfo>/<SignatureValue>
352
353
### Deletion Command Selectors
354
355 39 Shuo Chen
Deletion command selectors follows the [repo command selectors](http://redmine.named-data.net/projects/repo-ng/wiki/Repo_Protocol_Specification#Repo-Command-Selectors).
356
357 19 Shuo Chen
### Deletion status check
358
359 30 Shuo Chen
During the deletion progress, the requester could send deletion status check command to check the status of the deletion progress. This status check command is also signed interest. The semantics of deletion status check command follows the format of repo command. ``<command verb>`` equals to ``delete check``. For example:
360 1 Shuo Chen
361 34 Shuo Chen
    /ucla/cs/repo/delete/<RepoCommandParameters>/<timestamp>/<random-value>/<SignatureInfo>/<SignatureValue>
362 19 Shuo Chen
363 41 Shuo Chen
### Deletion Check Command Selectors
364
365
Selectors are not supported. Repo will ignore selectors when processing deletion status check command.
366
367 19 Shuo Chen
### Deletion status response
368
369 30 Shuo Chen
This delete status data object could be the response data packet of both delete command and delete check command. The name of data packet is the same as that of command.The deletion status is enocoded as a TLV echoded block in content block in data packet. The TLV block format follows RepoCommandResponse.
370 19 Shuo Chen
371
StatusCode indicates the status of deletion. DeleteSum is the sum of the deletion. DeleteNum represents how many data has been deleted.
372
373
StatusCode Definition:
374
375
``StatusCode`` | Description
376
-------------- | --------------------------------------
377
``200``        | All the data has been deleted
378
``300``        | This deletion is in progress
379
``401``        | This deletion check is invalidated
380
``404``        | No such this deletion is in progress
381 30 Shuo Chen
``505``        | status check command is not supported
382 19 Shuo Chen
383 17 Shuo Chen
### Protocol process
384 1 Shuo Chen
385 21 Alex Afanasyev
1. start to authorize the command; if authorization does not fail immediately, go to step 3
386 1 Shuo Chen
387 21 Alex Afanasyev
2. send a negative response indicating authorization failure, and abort these steps
388 1 Shuo Chen
389 21 Alex Afanasyev
3. check whether selectors are presented. If selectors are presented, go to step 4, or select the data packet of the exact name of command. If some data is selected, go to step 6. If no data selected, go to step 5
390 1 Shuo Chen
391 21 Alex Afanasyev
4. select all the data packets that comforms to data name of the command and the selectors. If some data packets are selected, go to step 6. Or no data selected, go to step 5.
392 1 Shuo Chen
393 21 Alex Afanasyev
5. send a negative response and abort the delete transaction.
394 1 Shuo Chen
395 21 Alex Afanasyev
6. wait for authorization completion
396 1 Shuo Chen
397 21 Alex Afanasyev
7. if authorization fails, go to step 2
398 1 Shuo Chen
399 21 Alex Afanasyev
8. start to delete the selected data. If deletion success, return positive reponse, or return negative response and abort the deletion.
400 19 Shuo Chen
401 21 Alex Afanasyev
Implementation MAY publish a notification of status regarding delete progress. The process of status check is as follows:
402 19 Shuo Chen
403 21 Alex Afanasyev
1. start to authorize the delete status command
404 19 Shuo Chen
405 21 Alex Afanasyev
2. send a negative response indicating authorization failure, and abort these steps
406 19 Shuo Chen
407 21 Alex Afanasyev
3. start to check the progress of the delete with the data name in the command. If no such progress is found, go to 4. or go to 5.
408 19 Shuo Chen
409 21 Alex Afanasyev
4. reponse status with status code of 404
410
411
5. ckeck te status of delete. return the status data content
412 19 Shuo Chen
413
### Protocol diagram:
414
415
    Requester                     Repo 
416
        |                           |                                 
417
        |                           |                                 
418
      +---+  Delete command       +---+                               
419
      |   | --------------------> |   |                               
420
      +---+                       +---+                               
421
        |                           |                                 
422
        |                           |                                 
423
        |                           |                                 
424
      +---+   Status interest     +---+                               
425
      |   | --------------------> |   |                               
426
      +---+                       |   |                               
427
        |                         |   |                               
428
      +---+    Status response    |   |                               
429
      |   | <==================== |   |                               
430
      +---+                       +---+                               
431
        |                           |                                 
432
        |                           |                                 
433
        |                           |                                 
434
      +---+   Confirm Deletion    +---+                               
435
      |   | <==================== |   |                               
436
      +---+   Reject command      +---+                               
437
        |     (with status code)    |    
438 20 Alex Afanasyev
        |                           |