Project

General

Profile

Repo Protocol Specification » History » Version 49

Shuo Chen, 02/24/2014 01:00 PM

1 1 Shuo Chen
Repo Protocol Specification
2
===========================
3
4 36 Shuo Chen
A Repo supports the network by preserving content and responding to Interests requesting content that it holds. A Repo can exist in any node, and is recommended if applications in that node need to preserve data. The NDN repo protocol is a specification of repo operations including reading, insertion and deletion of data objects in repo.
5 2 Shuo Chen
6
Repo semantics is based on [Command Interests](http://redmine.named-data.net/projects/nfd/wiki/Command_Interests) with signed commponents an the end of the name and basic common semantics of NDN. 
7
8 11 Shuo Chen
When some operation of repo including insertion and deletion of data objects is requested, a command interest is sent. The command interest is the interest as command of insertion and deletion and signed as form of command interest for access control. The repo will response the command with the data object.
9 4 Shuo Chen
10
The repo protocol can be categorized in three sections of reading, insertion and deletion of data objects.
11
12
## Repo Reading Specification
13
14 36 Shuo Chen
Repo registers prefixes of data objects it holds into NDN fowarding deamon and the repo will respond the data with such prefixes.
15 6 Shuo Chen
16 11 Shuo Chen
A standard interest is used to fetch content from the repo. The repo will respond when the name of the interest matches the prefix it registered in NFD. If the content in repo matches the interests, it will respond with the data object. When the interest is not matched, it will not respond.
17 6 Shuo Chen
18
The protocol is listed as bellow
19
20
If there is a matching data object:
21
22 7 Shuo Chen
    Requester                     Repo
23
        |                           |
24
        |                           |
25
        |         Interest          |
26
     t1 |-------------------------->|
27
        |                           |
28
        |        Data Object        |
29
     t2 |<==========================|
30 1 Shuo Chen
        |                           |
31 7 Shuo Chen
        |                           |
32 1 Shuo Chen
        |                           |
33 7 Shuo Chen
34
If there is no matching data object:
35 6 Shuo Chen
36
    Requester                     Repo
37
        |                           |
38 1 Shuo Chen
        |                           |
39 7 Shuo Chen
        |         Interest          |
40
     t1 |-------------------------->|
41
        |                           |
42
        |                           |
43
        |                           |
44
45 30 Shuo Chen
## Repo Command
46
For insertion, deletion and other operations of repo, these commands are encoded in the form of [sigend command interest](http://redmine.named-data.net/projects/nfd/wiki/Command_Interests). The semantics of repo command interest is as follows:
47 14 Shuo Chen
48 8 Shuo Chen
The name semantics is defined to have following components:
49 1 Shuo Chen
50 17 Shuo Chen
* ``<name of repo>`` refers to specific name of repo
51 30 Shuo Chen
* ``<command verb>`` refers to the name of command
52
* ``<RepoCommandParameters>`` refers to parameters of repo command
53 1 Shuo Chen
54 38 Shuo Chen
The following components are components of singed interest for access control:
55 1 Shuo Chen
56
* ``<timestamp>``
57 8 Shuo Chen
* ``<random-value>``
58
* ``<SignatureInfo>``
59
* ``<SignatureValue>``
60
61 30 Shuo Chen
For name of the repo /ucla/cs/repo/, the command will be defined as this:
62 33 Shuo Chen
63 30 Shuo Chen
    /ucla/cs/repo/<command verb>/<RepoCommandParameters>/<timestamp>/<random-value>/<SignatureInfo>/<SignatureValue>
64 14 Shuo Chen
65 30 Shuo Chen
### RepoCommandParameters
66 29 Shuo Chen
67 48 Shuo Chen
    RepoCommandParameters ::= REPOCOMMANDPARAMETERS-TYPE TLV-LENGTH
68 1 Shuo Chen
                               Name?
69 42 Shuo Chen
                               Selectors?
70 1 Shuo Chen
                               StartBlockId?
71
                               EndBlockId?
72 29 Shuo Chen
73 48 Shuo Chen
    Name                  ::= NAME-TYPE TLV-LENGTH NameComponent*
74
    NameComponent         ::= NAME-COMPONENT-TYPE TLV-LENGTH BYTE+
75 43 Shuo Chen
76
    Selectors             ::= SELECTORS-TYPE TLV-LENGTH
77
                               MinSuffixComponents?
78
                               MaxSuffixComponents?
79
                               PublisherPublicKeyLocator?
80
                               Exclude?
81 47 Shuo Chen
                               ChildSelector?
82 43 Shuo Chen
83
    MinSuffixComponents   ::= MIN-SUFFIX-COMPONENTS-TYPE TLV-LENGTH
84
                               nonNegativeInteger
85
86
    MaxSuffixComponents   ::= MAX-SUFFIX-COMPONENTS-TYPE TLV-LENGTH
87
                               nonNegativeInteger
88
89
    PublisherPublicKeyLocator ::= KeyLocator
90
91
    Exclude               ::= EXCLUDE-TYPE TLV-LENGTH Any? (NameComponent (Any)?)+
92
    Any                   ::= ANY-TYPE TLV-LENGTH(=0)
93 47 Shuo Chen
94
    ChildSelector         ::= CHILD-SELECTOR-TYPE TLV-LENGTH
95
                               nonNegativeInteger
96 1 Shuo Chen
    
97
    StartBlockId          ::= STARTBLOCKID-TYPE TLV-LENGTH
98 49 Shuo Chen
                               nonNegativeInteger
99 1 Shuo Chen
    
100
    EndBlockId            ::= ENDBLOCKID-TYPE TLV-LENGTH
101 49 Shuo Chen
                               nonNegativeInteger
102 1 Shuo Chen
103 45 Shuo Chen
#### Name
104
105
106 44 Shuo Chen
#### Repo Command Selectors
107 38 Shuo Chen
108
Repo command supports parts of interest selectors of interest to indicate which contents to process. The definition of standard NDN selectors is described in [NDN Selectors Doc](http://named-data.net/doc/ndn-tlv/interest.html#selectors). The concrete definitions of both standard NDN selectors and repo command selectors are the same. The difference between standard NDN interest and Repo Command interest that, the standard NDN selectors just matches **one** data packet that conforms to the selector conditions, but repo command selectors would matches **any** data packets. For example, if Interest is expressed for /ndn/edu and Exclude specifies one name component ucla, in standard NDN interest, the data producers will first exclude all the data packets with prefix /ndn/edu but not /ndn/edu/ucla, and then just selects one data packet. In repo command interest, it will select all the data packets with prefix /ndn/edu but not /ndn/edu/ucla.
109
110
Repo command supports parts of standard NDN interests including MinSuffixComponents, MaxSuffixComponents, PublisherPublicKeyLocator, Exclude. If command contains other selectors, repo will ignore these not supported selectors. In addition, selectors are just supported in delete command. If other commands contains selectors, repo will ignore selectors when processing these commands.
111
112
The form of selectors is as follows:
113
114
    Selectors             ::= SELECTORS-TYPE TLV-LENGTH
115
                               MinSuffixComponents?
116
                               MaxSuffixComponents?
117
                               PublisherPublicKeyLocator?
118
                               Exclude?
119
    
120
    MinSuffixComponents   ::= MIN-SUFFIX-COMPONENTS-TYPE TLV-LENGTH
121
                               nonNegativeInteger
122
123
    MaxSuffixComponents   ::= MAX-SUFFIX-COMPONENTS-TYPE TLV-LENGTH
124
                               nonNegativeInteger
125
126
    PublisherPublicKeyLocator ::= KeyLocator
127
    
128
    Exclude               ::= EXCLUDE-TYPE TLV-LENGTH Any? (NameComponent (Any)?)+
129 1 Shuo Chen
    Any                   ::= ANY-TYPE TLV-LENGTH(=0)
130 46 Shuo Chen
131
#### StartBlockId, EndBlockId
132 38 Shuo Chen
133 37 Shuo Chen
### Repo trust model
134
135
The trust model of repo depends on people who deploy the repo service, such as PKI. Repo can specify their own verification policies, and data consumers can specify their own trust anchors. The NDN [FAQ](http://named-data.net/project/faq/#How_does_NDN8217s_8220trust_management8221_work) shows how NDN trust managment works.
136
137 30 Shuo Chen
## Repo Command Response
138 1 Shuo Chen
139 30 Shuo Chen
Repo command response is the response data packet of repo command interest. The response contains statuscode to indicate the status of command process and other information. A TLV-encoded block called ``RepoCommandResponse`` is encoded in content of the data packet.
140 1 Shuo Chen
141 30 Shuo Chen
    RepoCommandResponse   ::= INSERTSTATUS-TYPE TLV-LENGTH
142 1 Shuo Chen
                               StatusCode?
143
                               StartBlockId?
144
                               EndBlockId?
145
                               InsertNum?
146 30 Shuo Chen
                               DeleteNum?
147 1 Shuo Chen
148
    StatusCode            ::= STATUSCODE-TYPE TLV-LENGTH
149
                                nonNegativeInteger    
150
151
    StartBlockId          ::= STARTBLOCKID-TYPE TLV-LENGTH
152
                                nonNegativeInteger
153
    
154
    EndBlockId            ::= ENDBLOCKID-TYPE TLV-LENGTH
155
                                nonNegativeInteger
156 17 Shuo Chen
157
    InsertNum             ::= INSERTNUM-TYPE TLV-LENGTH
158
                                nonNegativeInteger
159
160 30 Shuo Chen
    DeleteNum             ::= DELETENUM-TYPE TLV-LENGTH
161
                                nonNegativeInteger
162 17 Shuo Chen
163 30 Shuo Chen
## Repo Insertion Specification
164
165
Insertion specification is for some client and application to insert data objects into certain NDN repo. A command interest as insertion command will be sent to the repo. This command interest is a signed interest and will be validated with access control policy defined by the repo. When the interested is validated and name of the data is not existed in the repo. The repository will reponse with a data object containing OK status and start to send the interest to fetch the data to insert.
166
167
Segmented data insertion is also supported in the insertion protocol. Segmantation info is defined as a TLV-encoded SegmantationInfo block. If the content is segmented, the final segment id will be encoded in this block.
168
169
### Insertion command interest semantics
170
171
The name semantics is follows the format of the repo command. The <command verb> is defined as ``insert``.
172
For example, for ``<name of repo>`` as ``/ucla/cs/repo``, the following is an example:
173 1 Shuo Chen
174 31 Shuo Chen
    /ucla/cs/repo/insert/<RepoCommandParameters>/<timestamp>/<random-value>/<SignatureInfo>/<SignatureValue>
175 30 Shuo Chen
176
### Insertion status check
177
178
During the insertion progress, the requester could send insertion status check command to check the status of the insertion progress. This status check command is also signed interest. The semantics of insertion status check command is shown as follows:
179
180
<command verb> equals to ``insert check``. For example:
181 1 Shuo Chen
182 32 Shuo Chen
    /ucla/cs/repo/insert check/<RepoCommandParameters>/<timestamp>/<random-value>/<SignatureInfo>/<SignatureValue>
183 30 Shuo Chen
184 40 Shuo Chen
### Insert and Insert check command seletors
185
186
Selectors are not supported in these commands. Repo will ignore selectors when processing these commands.
187
188 30 Shuo Chen
### Insertatation status response
189
190
This insert status data object could be the response data object of both insert command and insert check command. It will follow the format of repo command response.
191
192
StatusCode indicates the status of insertion. InsertNum represents how many data has been intersted into the repo. StartBlockId and EndBlockId is the start and end segment id of data inserted. InsertNum is the how many data segments have been inserted.
193
194
For insert cammand, status code will be set according to definition below, StartBlockId and EndBlockId will be set according to the insert command.
195
196
For insert check command, status code will be set according to definition below, StartBlockId and EndBlockId will be set according to the insert command, insertnum will be set according to insertion progress.
197
198 1 Shuo Chen
StatusCode Definition:
199 17 Shuo Chen
200
``StatusCode`` | Description
201
-------------- | ----------------------------------------------
202 19 Shuo Chen
``100``        | The command is OK. can start to fetch the data
203
``200``        | All the data has been inserted
204 17 Shuo Chen
``300``        | This insertion is in progress
205
``401``        | This insertion check is invalidated
206
``404``        | No such this insertion is in progress
207 30 Shuo Chen
``505``        | status check command is not supported
208 17 Shuo Chen
209 22 Alex Afanasyev
### Protocol Process
210
211 20 Alex Afanasyev
1. start to authorize the command; if authorization does not fail immediately, go to step 3
212 17 Shuo Chen
213 20 Alex Afanasyev
2. send a negative response indicating authorization failure, and abort these steps
214 14 Shuo Chen
215 20 Alex Afanasyev
3. if both StartBlockId and EndBlockId are missing, go to step 6
216 14 Shuo Chen
217 20 Alex Afanasyev
4. if both StartBlockId and EndBlockId are present, and StartBlockId is less than or equal to EndBlockId, go to step 6
218 14 Shuo Chen
219 20 Alex Afanasyev
5. send a negative response indicating malformed command, and abort these steps
220 14 Shuo Chen
221 20 Alex Afanasyev
6. wait for authorization completion
222 14 Shuo Chen
223 20 Alex Afanasyev
7. if authorization fails, go to step 2
224 14 Shuo Chen
225 20 Alex Afanasyev
8. send a positive response indicating insert is in progress
226 14 Shuo Chen
227 20 Alex Afanasyev
9. if both StartBlockId and EndBlockId are present, go to step 15
228 14 Shuo Chen
229 20 Alex Afanasyev
10. start to retrieve Name
230 14 Shuo Chen
231 20 Alex Afanasyev
11. wait for retrieval completion
232 14 Shuo Chen
233 20 Alex Afanasyev
12. if retrieval fails, abort these steps
234 14 Shuo Chen
235 20 Alex Afanasyev
13. store retrieved Data packet
236 14 Shuo Chen
237 20 Alex Afanasyev
14. abort these steps
238 14 Shuo Chen
239 20 Alex Afanasyev
15. append StartBlockId to Name
240 1 Shuo Chen
241 20 Alex Afanasyev
16. start to retrieve Name
242 14 Shuo Chen
243 20 Alex Afanasyev
17. wait for retrieval completion
244 14 Shuo Chen
245 20 Alex Afanasyev
18. if retrieval fails, abort these steps
246 14 Shuo Chen
247 20 Alex Afanasyev
19. store retrieved Data packet
248 15 Shuo Chen
249 20 Alex Afanasyev
20. if retrieved Data packet contains FinalBlockId, and FinalBlockId is less than EndBlockId, let EndBlockId be FinalBlockId
250 15 Shuo Chen
251 20 Alex Afanasyev
21. if the last component of Name is greater than or equal to EndBlockId, abort these steps
252 15 Shuo Chen
253 20 Alex Afanasyev
22. increment the last component of Name
254 15 Shuo Chen
255 20 Alex Afanasyev
23. go to step 16
256 15 Shuo Chen
257 20 Alex Afanasyev
Implementation MAY pipeline the Interests.
258 1 Shuo Chen
259 20 Alex Afanasyev
### Repo command progress report
260 16 Shuo Chen
261 20 Alex Afanasyev
Implementation MAY publish a notification of status regarding insert progress. The process of status check is as follows:
262 18 Shuo Chen
263 20 Alex Afanasyev
1. start to authorize the insert status command
264 1 Shuo Chen
265 20 Alex Afanasyev
2. send a negative response indicating authorization failure, and abort these steps
266 18 Shuo Chen
267 20 Alex Afanasyev
3. start to check the progress of the insert with the data name in the command. If no such progress is found, go to 4. or go to 5.
268 18 Shuo Chen
269 20 Alex Afanasyev
4. reponse status with status code of 404
270 18 Shuo Chen
271 20 Alex Afanasyev
5. ckeck te status of insertion. return the status data content
272
273 17 Shuo Chen
### Protocol diagram:
274 10 Alex Afanasyev
275
    Requester                     Repo                          Data producer
276
        |                           |                                 |
277
        |                           |                                 |
278
      +---+  Insert command       +---+                               |
279
      |   | --------------------> |   |                               |
280
      +---+                       |   |                               |
281
        |                         |   |                               |
282
      +---+   Confirm start       |   |                               |
283
      |   | <==================== |   |                               |
284
      +---+   Reject command      +---+                               |
285
        |     (with status code)    |                                 |
286
        |                         +---+     Interest for Data       +---+
287 1 Shuo Chen
        |                         |   | --------------------------> |   |
288
        |                         +---+                             |   |
289 10 Alex Afanasyev
        |                           |                               |   |
290
        |                         +---+       Data segment          |   |
291
        |                         |   | <========================== |   |
292
        |                         +---+                             +---+
293
        |                           |                                 |
294
        |                           ~                                 ~
295
        |                           ~                                 ~
296
        |                           |                                 |
297
        |                         +---+     Interest for Data       +---+
298
        |                         |   | --------------------------> |   |
299
        |                         +---+                             |   |
300 1 Shuo Chen
        |                           |                               |   |
301
        |                         +---+       Data segment          |   |
302
        |                         |   | <========================== |   |
303
        |                         +---+                             +---+
304
        |                           |                                 |
305
        |                           |                                 |
306 17 Shuo Chen
        |                           ~                                 ~
307 1 Shuo Chen
        |                           ~                                 ~
308
        |                           |                                 |
309 30 Shuo Chen
        |                           |                                 |
310 19 Shuo Chen
        |                           |                                 |
311 30 Shuo Chen
      +---+   Status interest     +---+                               |
312 19 Shuo Chen
      |   | --------------------> |   |                               |
313
      +---+                       |   |                               |
314
        |                         |   |                               |
315
      +---+    Status response    |   |                               |
316
      |   | <==================== |   |                               |
317
      +---+                       +---+                               |
318
        |                           |                                 |
319
        |                           |                                 |
320
321
## Repo Deletion Specification
322
323
Deletion of one content object or content objects under certain prefix are both supported in repo specification. Selectors are used to select multiple content objects. These selectors are different from conventional [selectors](http://named-data.net/doc/ndn-tlv/interest.html#selectors) of interest packet. The conventional selecors of interest would select one data packet of repo. Selecors of delete command will return any data packets which applies for these selectors.
324
325
### Deletion command interest semantics
326
327 1 Shuo Chen
Deletion command interest follows the format of the repo command. For example:
328 19 Shuo Chen
329
    /ucla/cs/repo/delete/<RepoCommandParameters>/<timestamp>/<random-value>/<SignatureInfo>/<SignatureValue>
330
331
### Deletion Command Selectors
332
333 39 Shuo Chen
Deletion command selectors follows the [repo command selectors](http://redmine.named-data.net/projects/repo-ng/wiki/Repo_Protocol_Specification#Repo-Command-Selectors).
334
335 19 Shuo Chen
### Deletion status check
336
337 30 Shuo Chen
During the deletion progress, the requester could send deletion status check command to check the status of the deletion progress. This status check command is also signed interest. The semantics of deletion status check command follows the format of repo command. ``<command verb>`` equals to ``delete check``. For example:
338 1 Shuo Chen
339 34 Shuo Chen
    /ucla/cs/repo/delete/<RepoCommandParameters>/<timestamp>/<random-value>/<SignatureInfo>/<SignatureValue>
340 19 Shuo Chen
341 41 Shuo Chen
### Deletion Check Command Selectors
342
343
Selectors are not supported. Repo will ignore selectors when processing deletion status check command.
344
345 19 Shuo Chen
### Deletion status response
346
347 30 Shuo Chen
This delete status data object could be the response data packet of both delete command and delete check command. The name of data packet is the same as that of command.The deletion status is enocoded as a TLV echoded block in content block in data packet. The TLV block format follows RepoCommandResponse.
348 19 Shuo Chen
349
StatusCode indicates the status of deletion. DeleteSum is the sum of the deletion. DeleteNum represents how many data has been deleted.
350
351
StatusCode Definition:
352
353
``StatusCode`` | Description
354
-------------- | --------------------------------------
355
``200``        | All the data has been deleted
356
``300``        | This deletion is in progress
357
``401``        | This deletion check is invalidated
358
``404``        | No such this deletion is in progress
359 30 Shuo Chen
``505``        | status check command is not supported
360 19 Shuo Chen
361 17 Shuo Chen
### Protocol process
362 1 Shuo Chen
363 21 Alex Afanasyev
1. start to authorize the command; if authorization does not fail immediately, go to step 3
364 1 Shuo Chen
365 21 Alex Afanasyev
2. send a negative response indicating authorization failure, and abort these steps
366 1 Shuo Chen
367 21 Alex Afanasyev
3. check whether selectors are presented. If selectors are presented, go to step 4, or select the data packet of the exact name of command. If some data is selected, go to step 6. If no data selected, go to step 5
368 1 Shuo Chen
369 21 Alex Afanasyev
4. select all the data packets that comforms to data name of the command and the selectors. If some data packets are selected, go to step 6. Or no data selected, go to step 5.
370 1 Shuo Chen
371 21 Alex Afanasyev
5. send a negative response and abort the delete transaction.
372 1 Shuo Chen
373 21 Alex Afanasyev
6. wait for authorization completion
374 1 Shuo Chen
375 21 Alex Afanasyev
7. if authorization fails, go to step 2
376 1 Shuo Chen
377 21 Alex Afanasyev
8. start to delete the selected data. If deletion success, return positive reponse, or return negative response and abort the deletion.
378 19 Shuo Chen
379 21 Alex Afanasyev
Implementation MAY publish a notification of status regarding delete progress. The process of status check is as follows:
380 19 Shuo Chen
381 21 Alex Afanasyev
1. start to authorize the delete status command
382 19 Shuo Chen
383 21 Alex Afanasyev
2. send a negative response indicating authorization failure, and abort these steps
384 19 Shuo Chen
385 21 Alex Afanasyev
3. start to check the progress of the delete with the data name in the command. If no such progress is found, go to 4. or go to 5.
386 19 Shuo Chen
387 21 Alex Afanasyev
4. reponse status with status code of 404
388
389
5. ckeck te status of delete. return the status data content
390 19 Shuo Chen
391
### Protocol diagram:
392
393
    Requester                     Repo 
394
        |                           |                                 
395
        |                           |                                 
396
      +---+  Delete command       +---+                               
397
      |   | --------------------> |   |                               
398
      +---+                       +---+                               
399
        |                           |                                 
400
        |                           |                                 
401
        |                           |                                 
402
      +---+   Status interest     +---+                               
403
      |   | --------------------> |   |                               
404
      +---+                       |   |                               
405
        |                         |   |                               
406
      +---+    Status response    |   |                               
407
      |   | <==================== |   |                               
408
      +---+                       +---+                               
409
        |                           |                                 
410
        |                           |                                 
411
        |                           |                                 
412
      +---+   Confirm Deletion    +---+                               
413
      |   | <==================== |   |                               
414
      +---+   Reject command      +---+                               
415
        |     (with status code)    |    
416 20 Alex Afanasyev
        |                           |