Project

General

Profile

Repo Protocol Specification » History » Version 27

Alex Afanasyev, 02/20/2014 06:12 PM

1 1 Shuo Chen
Repo Protocol Specification
2
===========================
3
4 2 Shuo Chen
Repo is persistent storage of data objects model in NDN. The NDN repo protocol is a specification of repo operations including reading, insertion and deletion of data objects in repo.
5
6
Repo semantics is based on [Command Interests](http://redmine.named-data.net/projects/nfd/wiki/Command_Interests) with signed commponents an the end of the name and basic common semantics of NDN. 
7
8 11 Shuo Chen
When some operation of repo including insertion and deletion of data objects is requested, a command interest is sent. The command interest is the interest as command of insertion and deletion and signed as form of command interest for access control. The repo will response the command with the data object.
9 4 Shuo Chen
10
The repo protocol can be categorized in three sections of reading, insertion and deletion of data objects.
11
12
## Repo Reading Specification
13
14 11 Shuo Chen
The repo could be viewed as the data container of the data objects. It will register some specific prefixes of data objects it holds into NFD (NDN Forwarding Deamon) and the repo will respond the data with such prefixes.
15 6 Shuo Chen
16 11 Shuo Chen
A standard interest is used to fetch content from the repo. The repo will respond when the name of the interest matches the prefix it registered in NFD. If the content in repo matches the interests, it will respond with the data object. When the interest is not matched, it will not respond.
17 6 Shuo Chen
18
The protocol is listed as bellow
19
20
If there is a matching data object:
21
22 7 Shuo Chen
    Requester                     Repo
23
        |                           |
24
        |                           |
25
        |         Interest          |
26
     t1 |-------------------------->|
27
        |                           |
28
        |        Data Object        |
29
     t2 |<==========================|
30 1 Shuo Chen
        |                           |
31 7 Shuo Chen
        |                           |
32 1 Shuo Chen
        |                           |
33 7 Shuo Chen
34
If there is no matching data object:
35 6 Shuo Chen
36
    Requester                     Repo
37
        |                           |
38 1 Shuo Chen
        |                           |
39 7 Shuo Chen
        |         Interest          |
40
     t1 |-------------------------->|
41
        |                           |
42
        |                           |
43
        |                           |
44
45 8 Shuo Chen
## Repo Insertion Specification
46
47 11 Shuo Chen
Insertion specification is for some client and application to insert data objects into certain NDN repo. A command interest as insertion command will be sent to the repo. This command interest is a signed interest and will be validated with access control policy defined by the repo. When the interested is validated and name of the data is not existed in the repo. The repository will reponse with a data object containing OK status and start to send the interest to fetch the data to insert.
48 8 Shuo Chen
49 19 Shuo Chen
Segmented data insertion is also supported in the insertion protocol. Segmantation info is defined as a TLV-encoded SegmantationInfo block. If the content is segmented, the final segment id will be encoded in this block.
50 14 Shuo Chen
51 8 Shuo Chen
### Insertion command interest semantics
52 1 Shuo Chen
53
The name semantics is defined to have following components:
54
55 17 Shuo Chen
* ``<name of repo>`` refers to specific name of repo
56 11 Shuo Chen
* ``insert`` refers to insert data objects
57 1 Shuo Chen
* ``<data name>`` refers to specific name of data that will be inserted
58
59 8 Shuo Chen
The folloing components are components of singed interest for access control:
60
61
* ``<timestamp>``
62
* ``<random-value>``
63
* ``<SignatureInfo>``
64
* ``<SignatureValue>``
65
66 17 Shuo Chen
<name of repo> and <data name> can be divided by serveral continious ``/`` according to specific defination.
67
For example, for ``<name of repo>`` as ``/ucla/cs/repo`` and ``<data name>`` as ``/vedio/Forrest Gump``, the data object is defined as following:
68 8 Shuo Chen
69 27 Alex Afanasyev
    /ucla/cs/repo/insert/<RepoCommandParameters>/........................
70 14 Shuo Chen
71
### SegmantationInfo
72
73
    SegmantationInfo      ::= SEGMENTATIONINFO-TYPE TLV-LENGTH
74
                               StartBlockId?
75 23 Shuo Chen
                               EndBlockId?
76 14 Shuo Chen
    
77
    StartBlockId          ::= STARTBLOCKID-TYPE TLV-LENGTH
78
                                nonNegativeInteger
79 1 Shuo Chen
    
80 24 Shuo Chen
    EndBlockId            ::= ENDBLOCKID-TYPE TLV-LENGTH
81 1 Shuo Chen
                                nonNegativeInteger
82
83 17 Shuo Chen
### Insertion status check
84 1 Shuo Chen
85 19 Shuo Chen
During the insertion progress, the requester could send insertion status check command to check the status of the insertion progress. This status check command is also signed interest. The semantics of insertion status check command is shown as follows:
86 17 Shuo Chen
87
The name semantics is defined to have following components:
88
89
* ``<name of repo>`` refers to specific name of repo
90
* ``insert check`` refers to insert data objects
91 26 Shuo Chen
* ``<insert option>`` refers to insert options
92 17 Shuo Chen
* ``<data name>`` refers to specific name of data that will be inserted
93
94
The folloing components are components of singed interest for access control:
95
96
* ``<timestamp>``
97
* ``<random-value>``
98
* ``<SignatureInfo>``
99 1 Shuo Chen
* ``<SignatureValue>``
100
101 19 Shuo Chen
### Insertatation status response
102
103
This insert status data object could be the response data object of both insert command and insert check command. The name of data packet is the same as that of command. The insertion status is enocoded as a TLV echoded block in content block in data packet. The TLV block format is as follows:
104
105 17 Shuo Chen
    InsertStatus          ::= INSERTSTATUS-TYPE TLV-LENGTH
106
                               StatusCode?
107 1 Shuo Chen
                               StartBlockId?
108 25 Shuo Chen
                               EndBlockId?
109 17 Shuo Chen
                               InsertNum?
110
111
    StatusCode            ::= STATUSCODE-TYPE TLV-LENGTH
112
                                nonNegativeInteger    
113
114
    StartBlockId          ::= STARTBLOCKID-TYPE TLV-LENGTH
115 1 Shuo Chen
                                nonNegativeInteger
116 17 Shuo Chen
    
117 25 Shuo Chen
    EndBlockId            ::= ENDBLOCKID-TYPE TLV-LENGTH
118 17 Shuo Chen
                                nonNegativeInteger
119 1 Shuo Chen
120 17 Shuo Chen
    InsertNum             ::= INSERTNUM-TYPE TLV-LENGTH
121
                                nonNegativeInteger
122
123 19 Shuo Chen
StatusCode indicates the status of insertion. InsertNum represents how many data has been intersted into the repo.
124 1 Shuo Chen
125 17 Shuo Chen
StatusCode Definition:
126
127
``StatusCode`` | Description
128 19 Shuo Chen
-------------- | ----------------------------------------------
129
``100``        | The command is OK. can start to fetch the data
130 17 Shuo Chen
``200``        | All the data has been inserted
131
``300``        | This insertion is in progress
132
``401``        | This insertion check is invalidated
133
``404``        | No such this insertion is in progress
134
135 22 Alex Afanasyev
### Protocol Process
136
137 20 Alex Afanasyev
1. start to authorize the command; if authorization does not fail immediately, go to step 3
138 17 Shuo Chen
139 20 Alex Afanasyev
2. send a negative response indicating authorization failure, and abort these steps
140 14 Shuo Chen
141 20 Alex Afanasyev
3. if both StartBlockId and EndBlockId are missing, go to step 6
142 14 Shuo Chen
143 20 Alex Afanasyev
4. if both StartBlockId and EndBlockId are present, and StartBlockId is less than or equal to EndBlockId, go to step 6
144 14 Shuo Chen
145 20 Alex Afanasyev
5. send a negative response indicating malformed command, and abort these steps
146 14 Shuo Chen
147 20 Alex Afanasyev
6. wait for authorization completion
148 14 Shuo Chen
149 20 Alex Afanasyev
7. if authorization fails, go to step 2
150 14 Shuo Chen
151 20 Alex Afanasyev
8. send a positive response indicating insert is in progress
152 14 Shuo Chen
153 20 Alex Afanasyev
9. if both StartBlockId and EndBlockId are present, go to step 15
154 14 Shuo Chen
155 20 Alex Afanasyev
10. start to retrieve Name
156 14 Shuo Chen
157 20 Alex Afanasyev
11. wait for retrieval completion
158 14 Shuo Chen
159 20 Alex Afanasyev
12. if retrieval fails, abort these steps
160 14 Shuo Chen
161 20 Alex Afanasyev
13. store retrieved Data packet
162 14 Shuo Chen
163 20 Alex Afanasyev
14. abort these steps
164 14 Shuo Chen
165 20 Alex Afanasyev
15. append StartBlockId to Name
166 1 Shuo Chen
167 20 Alex Afanasyev
16. start to retrieve Name
168 14 Shuo Chen
169 20 Alex Afanasyev
17. wait for retrieval completion
170 14 Shuo Chen
171 20 Alex Afanasyev
18. if retrieval fails, abort these steps
172 14 Shuo Chen
173 20 Alex Afanasyev
19. store retrieved Data packet
174 15 Shuo Chen
175 20 Alex Afanasyev
20. if retrieved Data packet contains FinalBlockId, and FinalBlockId is less than EndBlockId, let EndBlockId be FinalBlockId
176 15 Shuo Chen
177 20 Alex Afanasyev
21. if the last component of Name is greater than or equal to EndBlockId, abort these steps
178 15 Shuo Chen
179 20 Alex Afanasyev
22. increment the last component of Name
180 15 Shuo Chen
181 20 Alex Afanasyev
23. go to step 16
182 15 Shuo Chen
183 20 Alex Afanasyev
Implementation MAY pipeline the Interests.
184 1 Shuo Chen
185 20 Alex Afanasyev
### Repo command progress report
186 16 Shuo Chen
187 20 Alex Afanasyev
Implementation MAY publish a notification of status regarding insert progress. The process of status check is as follows:
188 18 Shuo Chen
189 20 Alex Afanasyev
1. start to authorize the insert status command
190 1 Shuo Chen
191 20 Alex Afanasyev
2. send a negative response indicating authorization failure, and abort these steps
192 18 Shuo Chen
193 20 Alex Afanasyev
3. start to check the progress of the insert with the data name in the command. If no such progress is found, go to 4. or go to 5.
194 18 Shuo Chen
195 20 Alex Afanasyev
4. reponse status with status code of 404
196 18 Shuo Chen
197 20 Alex Afanasyev
5. ckeck te status of insertion. return the status data content
198
199 17 Shuo Chen
### Protocol diagram:
200 10 Alex Afanasyev
201
    Requester                     Repo                          Data producer
202
        |                           |                                 |
203
        |                           |                                 |
204
      +---+  Insert command       +---+                               |
205
      |   | --------------------> |   |                               |
206
      +---+                       |   |                               |
207
        |                         |   |                               |
208
      +---+   Confirm start       |   |                               |
209
      |   | <==================== |   |                               |
210
      +---+   Reject command      +---+                               |
211
        |     (with status code)    |                                 |
212
        |                         +---+     Interest for Data       +---+
213
        |                         |   | --------------------------> |   |
214
        |                         +---+                             |   |
215
        |                           |                               |   |
216
        |                         +---+       Data segment          |   |
217
        |                         |   | <========================== |   |
218
        |                         +---+                             +---+
219
        |                           |                                 |
220
        |                           ~                                 ~
221
        |                           ~                                 ~
222
        |                           |                                 |
223
        |                         +---+     Interest for Data       +---+
224
        |                         |   | --------------------------> |   |
225
        |                         +---+                             |   |
226
        |                           |                               |   |
227
        |                         +---+       Data segment          |   |
228
        |                         |   | <========================== |   |
229
        |                         +---+                             +---+
230
        |                           |                                 |
231
        |                           |                                 |
232
        |                           ~                                 ~
233
        |                           ~                                 ~
234
        |                           |                                 |
235
        |                           |                                 |
236
        |                           |                                 |
237
      +---+   Status interest     +---+                               |
238
      |   | --------------------> |   |                               |
239
      +---+                       |   |                               |
240 1 Shuo Chen
        |                         |   |                               |
241
      +---+    Status response    |   |                               |
242
      |   | <==================== |   |                               |
243
      +---+                       +---+                               |
244
        |                           |                                 |
245
        |                           |                                 |
246
247
## Repo Deletion Specification
248
249
Deletion of one content object or content objects under certain prefix are both supported in repo specification. Selectors are used to select multiple content objects. These selectors are different from conventional [selectors](http://named-data.net/doc/ndn-tlv/interest.html#selectors) of interest packet. The conventional selecors of interest would select one data packet of repo. Selecors of delete command will return any data packets which applies for these selectors.
250
251 19 Shuo Chen
### Deletion command interest semantics
252
253
The name semantics is defined to have following components:
254
255
* ``<name of repo>`` refers to specific name of repo
256
* ``delete`` refers to insert data objects
257
* ``<data name>`` refers to specific name of data that will be inserted
258
259
The folloing components are components of singed interest for access control:
260
261
* ``<timestamp>``
262
* ``<random-value>``
263
* ``<SignatureInfo>``
264
* ``<SignatureValue>``
265
266
<name of repo> and <data name> can be divided by serveral continious ``/`` according to specific defination.
267
For example, for ``<name of repo>`` as ``/ucla/cs/repo`` and ``<data name>`` as ``/vedio/Forrest Gump``, the data object is defined as following:
268
269
    /ucla/cs/repo/delete/vedio/Forrest Gump/<timestamp>/<random-value>/<SignatureInfo>/<SignatureValue>
270
271 17 Shuo Chen
### Deletion Command Selectors
272
273
    Selectors             ::= SELECTORS-TYPE TLV-LENGTH
274
                               MinSuffixComponents?
275
                               MaxSuffixComponents?
276
                               PublisherPublicKeyLocator?
277
                               Exclude?
278
    
279
    MinSuffixComponents   ::= MIN-SUFFIX-COMPONENTS-TYPE TLV-LENGTH
280
                               nonNegativeInteger
281
282
    MaxSuffixComponents   ::= MAX-SUFFIX-COMPONENTS-TYPE TLV-LENGTH
283
                               nonNegativeInteger
284
285
    PublisherPublicKeyLocator ::= KeyLocator
286
    
287 1 Shuo Chen
    Exclude               ::= EXCLUDE-TYPE TLV-LENGTH Any? (NameComponent (Any)?)+
288
    Any                   ::= ANY-TYPE TLV-LENGTH(=0)
289
290
The definitions of delete command selectors is the same as the selectors of conventional [selectors](http://named-data.net/doc/ndn-tlv/interest.html#selectors). The difference is that this delete command selectors will return any data packets that comforms to the selectors. If selectors are not presented, the repo will just delete the data of exact name. So, for example, if somebody wants to delete all the data under certain prefix, at least, MinSuffixComponents should be set to 0.
291
292 19 Shuo Chen
### Deletion status check
293
294
During the deletion progress, the requester could send deletion status check command to check the status of the deletion progress. This status check command is also signed interest. The semantics of deletion status check command is shown as follows:
295
296
The name semantics is defined to have following components:
297
298
* ``<name of repo>`` refers to specific name of repo
299
* ``delete check`` refers to insert data objects
300
* ``<data name>`` refers to specific name of data that will be inserted
301
302
The folloing components are components of singed interest for access control:
303
304
* ``<timestamp>``
305
* ``<random-value>``
306
* ``<SignatureInfo>``
307
* ``<SignatureValue>``
308
309
### Deletion status response
310
311
This delete status data object could be the response data packet of both delete command and delete check command. The name of data packet is the same as that of command.The deletion status is enocoded as a TLV echoded block in content block in data packet. The TLV block format is as follows:
312
313
    DeleteStatus          ::= DELETESTATUS-TYPE TLV-LENGTH
314
                               StatusCode?
315
                               DeleteSum?
316
                               DeleteNum?
317
318
    StatusCode            ::= STATUSCODE-TYPE TLV-LENGTH
319
                                nonNegativeInteger    
320
321
    DeleteSum             ::= INSERTNUM-TYPE TLV-LENGTH
322
                                nonNegativeInteger
323
324
    DeleteNum             ::= INSERTNUM-TYPE TLV-LENGTH
325
                                nonNegativeInteger
326
327
StatusCode indicates the status of deletion. DeleteSum is the sum of the deletion. DeleteNum represents how many data has been deleted.
328
329
StatusCode Definition:
330
331
``StatusCode`` | Description
332
-------------- | --------------------------------------
333
``200``        | All the data has been deleted
334
``300``        | This deletion is in progress
335
``401``        | This deletion check is invalidated
336
``404``        | No such this deletion is in progress
337
338 17 Shuo Chen
### Protocol process
339 1 Shuo Chen
340 21 Alex Afanasyev
1. start to authorize the command; if authorization does not fail immediately, go to step 3
341 1 Shuo Chen
342 21 Alex Afanasyev
2. send a negative response indicating authorization failure, and abort these steps
343 1 Shuo Chen
344 21 Alex Afanasyev
3. check whether selectors are presented. If selectors are presented, go to step 4, or select the data packet of the exact name of command. If some data is selected, go to step 6. If no data selected, go to step 5
345 1 Shuo Chen
346 21 Alex Afanasyev
4. select all the data packets that comforms to data name of the command and the selectors. If some data packets are selected, go to step 6. Or no data selected, go to step 5.
347 1 Shuo Chen
348 21 Alex Afanasyev
5. send a negative response and abort the delete transaction.
349 1 Shuo Chen
350 21 Alex Afanasyev
6. wait for authorization completion
351 1 Shuo Chen
352 21 Alex Afanasyev
7. if authorization fails, go to step 2
353 1 Shuo Chen
354 21 Alex Afanasyev
8. start to delete the selected data. If deletion success, return positive reponse, or return negative response and abort the deletion.
355 19 Shuo Chen
356 21 Alex Afanasyev
Implementation MAY publish a notification of status regarding delete progress. The process of status check is as follows:
357 19 Shuo Chen
358 21 Alex Afanasyev
1. start to authorize the delete status command
359 19 Shuo Chen
360 21 Alex Afanasyev
2. send a negative response indicating authorization failure, and abort these steps
361 19 Shuo Chen
362 21 Alex Afanasyev
3. start to check the progress of the delete with the data name in the command. If no such progress is found, go to 4. or go to 5.
363 19 Shuo Chen
364 21 Alex Afanasyev
4. reponse status with status code of 404
365
366
5. ckeck te status of delete. return the status data content
367 19 Shuo Chen
368
### Protocol diagram:
369
370
    Requester                     Repo 
371
        |                           |                                 
372
        |                           |                                 
373
      +---+  Delete command       +---+                               
374
      |   | --------------------> |   |                               
375
      +---+                       +---+                               
376
        |                           |                                 
377
        |                           |                                 
378
        |                           |                                 
379
      +---+   Status interest     +---+                               
380
      |   | --------------------> |   |                               
381
      +---+                       |   |                               
382
        |                         |   |                               
383
      +---+    Status response    |   |                               
384
      |   | <==================== |   |                               
385
      +---+                       +---+                               
386
        |                           |                                 
387
        |                           |                                 
388
        |                           |                                 
389
      +---+   Confirm Deletion    +---+                               
390
      |   | <==================== |   |                               
391
      +---+   Reject command      +---+                               
392
        |     (with status code)    |    
393 20 Alex Afanasyev
        |                           |