https://redmine.named-data.net/https://redmine.named-data.net/favicon.ico?14759811232017-12-20T23:07:07ZNDN project issue tracking systemNFD - Feature #4318: Content Store flush/erase commandhttps://redmine.named-data.net/issues/4318?journal_id=214652017-12-20T23:07:07ZDavide Pesavento
<ul><li><strong>Start date</strong> deleted (<del><i>10/02/2017</i></del>)</li></ul> NFD - Feature #4318: Content Store flush/erase commandhttps://redmine.named-data.net/issues/4318?journal_id=214682017-12-20T23:08:27ZDavide Pesavento
<ul><li><strong>Related to</strong> <i><a class="issue tracker-2 status-5 priority-2 priority-default closed" href="/issues/4050">Feature #4050</a>: Content Store Manager</i> added</li></ul> NFD - Feature #4318: Content Store flush/erase commandhttps://redmine.named-data.net/issues/4318?journal_id=222032018-02-08T17:49:57ZJunxiao Shi
<ul><li><strong>Status</strong> changed from <i>New</i> to <i>In Progress</i></li><li><strong>Assignee</strong> set to <i>Junxiao Shi</i></li><li><strong>Target version</strong> set to <i>v0.7</i></li><li><strong>% Done</strong> changed from <i>0</i> to <i>10</i></li></ul><p><a class="wiki-page" href="https://redmine.named-data.net/projects/nfd/wiki/CsMgmt">CsMgmt</a> rev9 and <a class="wiki-page" href="https://redmine.named-data.net/projects/nfd/wiki/ControlCommand">ControlCommand</a> rev37 introduce <code>cs/erase</code> command.</p>
NFD - Feature #4318: Content Store flush/erase commandhttps://redmine.named-data.net/issues/4318?journal_id=222142018-02-09T15:16:14ZDavide Pesavento
<ul></ul><p>Junxiao Shi wrote:</p>
<blockquote>
<p><a class="wiki-page" href="https://redmine.named-data.net/projects/nfd/wiki/CsMgmt">CsMgmt</a> rev9 and <a class="wiki-page" href="https://redmine.named-data.net/projects/nfd/wiki/ControlCommand">ControlCommand</a> rev37 introduce <code>cs/erase</code> command.</p>
</blockquote>
<p>In ControlParameters, can we give <code>NCsEntries</code> a more generic name, in case it's needed in the future for similar purposes but not related to the CS? Some suggestions are: <code>NEntries</code>, <code>Count</code>.</p>
<p>Other than that, LGTM.</p>
NFD - Feature #4318: Content Store flush/erase commandhttps://redmine.named-data.net/issues/4318?journal_id=222172018-02-10T08:05:56ZJunxiao Shi
<ul></ul><blockquote>
<p>In ControlParameters, can we give <code>NCsEntries</code> a more generic name, in case it's needed in the future for similar purposes but not related to the CS? Some suggestions are: <code>NEntries</code>, <code>Count</code>.</p>
</blockquote>
<p>0x87 is associated with <strong>NCsEntries</strong> tag, so it's most convenient to keep the existing tag. Other purposes may reuse 0x87 regardless of what it's called.</p>
NFD - Feature #4318: Content Store flush/erase commandhttps://redmine.named-data.net/issues/4318?journal_id=222332018-02-11T08:49:49ZJunxiao Shi
<ul></ul><p><a href="https://gerrit.named-data.net/4521">https://gerrit.named-data.net/4521</a> declares <code>cs/erase</code> command.<br>
<a class="wiki-page" href="https://redmine.named-data.net/projects/nfd/wiki/CsMgmt">CsMgmt</a> rev14 requires NCsEntries in request to be positive.</p>
NFD - Feature #4318: Content Store flush/erase commandhttps://redmine.named-data.net/issues/4318?journal_id=222392018-02-11T11:28:03ZDavide Pesavento
<ul></ul><p>Junxiao Shi wrote:</p>
<blockquote>
<blockquote>
<p>In ControlParameters, can we give <code>NCsEntries</code> a more generic name, in case it's needed in the future for similar purposes but not related to the CS? Some suggestions are: <code>NEntries</code>, <code>Count</code>.</p>
</blockquote>
<p>0x87 is associated with <strong>NCsEntries</strong> tag, so it's most convenient to keep the existing tag. Other purposes may reuse 0x87 regardless of what it's called.</p>
</blockquote>
<p>There is no "existing tag". You just added the assignment in r37 of <a class="wiki-page" href="https://redmine.named-data.net/projects/nfd/wiki/ControlCommand">ControlCommand</a>.<br>
There is no reason to tie this field to the CS. In the same spirit, the capacity field was named <code>Capacity</code> and not <code>CsCapacity</code>.</p>
NFD - Feature #4318: Content Store flush/erase commandhttps://redmine.named-data.net/issues/4318?journal_id=222502018-02-11T20:50:19ZJunxiao Shi
<ul></ul><blockquote>
<p>There is no "existing tag". You just added the assignment in r37 of <a class="wiki-page" href="https://redmine.named-data.net/projects/nfd/wiki/ControlCommand">ControlCommand</a>.</p>
</blockquote>
<p>The tag for 0x87 is NCsEntries and it existed in <a class="wiki-page" href="https://redmine.named-data.net/projects/nfd/wiki/ForwarderStatus">ForwarderStatus</a>.</p>
<blockquote>
<p>There is no reason to tie this field to the CS. In the same spirit, the capacity field was named <code>Capacity</code> and not <code>CsCapacity</code>.</p>
</blockquote>
<p>Logically each command has its own parameters type. There should be no consideration for other commands whatsoever in adding a field for cs/erase.<br>
Microsoft adCenter API declares a separate parameter class for each command, with automatic code generation out of WSDL. We should do the same for each command.</p>
NFD - Feature #4318: Content Store flush/erase commandhttps://redmine.named-data.net/issues/4318?journal_id=223212018-02-13T18:19:08ZDavide Pesavento
<ul></ul><p>Junxiao Shi wrote:</p>
<blockquote>
<p>The tag for 0x87 is NCsEntries and it existed in <a class="wiki-page" href="https://redmine.named-data.net/projects/nfd/wiki/ForwarderStatus">ForwarderStatus</a>.</p>
</blockquote>
<p>It's a different TLV element...</p>
<blockquote>
<blockquote>
<p>There is no reason to tie this field to the CS. In the same spirit, the capacity field was named <code>Capacity</code> and not <code>CsCapacity</code>.</p>
</blockquote>
<p>Logically each command has its own parameters type. There should be no consideration for other commands whatsoever in adding a field for cs/erase.<br>
Microsoft adCenter API declares a separate parameter class for each command, with automatic code generation out of WSDL. We should do the same for each command.</p>
</blockquote>
<p>How is this related to my comment? Yes, the current ControlParameters structure is ugly, but that's a separate issue that should be discussed/solved separately. It doesn't answer my concern. In fact, if "logically each command has its own parameters type", then there's yet another reason to <em>not</em> include "Cs" in the param name.</p>
NFD - Feature #4318: Content Store flush/erase commandhttps://redmine.named-data.net/issues/4318?journal_id=225922018-03-03T17:35:15ZAlex Afanasyev
<ul></ul><p>The spec needs to define how to remove "all" entries under the name. I don't think using max is a good option; I would use <code>0</code> that is currently prohibited.</p>
NFD - Feature #4318: Content Store flush/erase commandhttps://redmine.named-data.net/issues/4318?journal_id=225932018-03-03T17:41:04ZJunxiao Shi
<ul></ul><blockquote>
<p>The spec needs to define how to remove "all" entries under the name. I don't think using max is a good option; I would use <code>0</code> that is currently prohibited.</p>
</blockquote>
<p>The caller has to repeat the operation until <strong>NCsEntries</strong> in the response becomes zero. The loop will be implemented in nfdc.<br>
It's intentional to have an upper bound and disallow erasing too many entries at a time, because it would put too much stress on tables. This limitation can be lifted after <a class="issue tracker-2 status-2 priority-2 priority-default" title="Feature: Management thread (In Progress)" href="https://redmine.named-data.net/issues/4528">#4528</a>.</p>
NFD - Feature #4318: Content Store flush/erase commandhttps://redmine.named-data.net/issues/4318?journal_id=225952018-03-03T17:59:43ZDavide Pesavento
<ul></ul><p>Junxiao Shi wrote:</p>
<blockquote>
<p>This limitation can be lifted after <a class="issue tracker-2 status-2 priority-2 priority-default" title="Feature: Management thread (In Progress)" href="https://redmine.named-data.net/issues/4528">#4528</a>.</p>
</blockquote>
<p>Not really. The mgmt thread still has to lock the table to perform the erasure.</p>
NFD - Feature #4318: Content Store flush/erase commandhttps://redmine.named-data.net/issues/4318?journal_id=225962018-03-03T18:05:02ZJunxiao Shi
<ul></ul><blockquote>
<blockquote>
<p>This limitation can be lifted after <a class="issue tracker-2 status-2 priority-2 priority-default" title="Feature: Management thread (In Progress)" href="https://redmine.named-data.net/issues/4528">#4528</a>.</p>
</blockquote>
<p>Not really. The mgmt thread still has to lock the table to perform the erasure.</p>
</blockquote>
<p>Mgmt thread can erase CS entries in batches, and unlock the table in between.</p>
NFD - Feature #4318: Content Store flush/erase commandhttps://redmine.named-data.net/issues/4318?journal_id=225992018-03-03T18:23:54ZDavide Pesavento
<ul></ul><p>Junxiao Shi wrote:</p>
<blockquote>
<blockquote>
<blockquote>
<p>This limitation can be lifted after <a class="issue tracker-2 status-2 priority-2 priority-default" title="Feature: Management thread (In Progress)" href="https://redmine.named-data.net/issues/4528">#4528</a>.</p>
</blockquote>
<p>Not really. The mgmt thread still has to lock the table to perform the erasure.</p>
</blockquote>
<p>Mgmt thread can erase CS entries in batches, and unlock the table in between.</p>
</blockquote>
<p>That doesn't require a separate thread... Anyway, we're going off-topic.</p>
<p>Regardless of the forwarder-side upper bound on the number of entries erased with a single request, I think what Alex is saying is that there should be a way to specify "erase <strong>all entries</strong>" at the protocol level (which becomes "erase <em>as many as possible</em>" if the forwarder has an upper limit). One could use a very large number, or query the CS capacity first and use that as limit. Both approaches don't look very elegant though...</p>
NFD - Feature #4318: Content Store flush/erase commandhttps://redmine.named-data.net/issues/4318?journal_id=226012018-03-04T05:18:52ZJunxiao Shi
<ul></ul><blockquote>
<p>Regardless of the forwarder-side upper bound on the number of entries erased with a single request, there should be a way to specify "erase <strong>all entries</strong>" at the protocol level (which becomes "erase <em>as many as possible</em>" if the forwarder has an upper limit).</p>
</blockquote>
<p><a class="wiki-page" href="https://redmine.named-data.net/projects/nfd/wiki/CsMgmt">CsMgmt</a> rev15 specifies that omitted <em>NCsEntries</em> in a request indicates "no limit".<br>
This revision also removes the requirement of changing StatusCode to 206 when exceeding the limit, because the inclusion of <em>Capacity</em> in a response already indicates this condition.</p>
NFD - Feature #4318: Content Store flush/erase commandhttps://redmine.named-data.net/issues/4318?journal_id=226172018-03-05T16:28:00ZAlex Afanasyev
<ul><li><strong>Priority</strong> changed from <i>Normal</i> to <i>Urgent</i></li></ul><p>The merged ndn-cxx:commit:df5053824fc53eec6847067faa2da8b0ebb69917 breaks NFD. I have no clue how, but simply compiling ndn-cxx, NFD against it, and running NFD results in a segfault (assertion fault) with the last words:</p>
<pre><code>...
1520295590.549215 TRACE: [ndn.security.v2.ValidationState] ~ValidationState
1520295590.549512 DEBUG: [ndn.security.v2.Validator] > Start validating data /localhost/nfd/faces/update/h%06l%01%01p%01%01/%00%00%01a%F8%AD%BA%0F/%16%AE%29%A5%C1%BBZ%1E/%16%1C%1B%01%01%1C%17%07%15%08%04test%08%03KEY%08%08%D0%80%A8%A7Hr0%C6/%17%FD%01%00x%89%24%1C%3AK4%816%86%5B%B3%9Bh%F8%99DHU%F3%19A%3Eh%F9%BFLTX%92%9B%06%BE%9E%60%81%7B%B6%5D%96%F9%27%22e%88%88%40%EAr%EF%A1%20%E5%AA7Z%FC%C6%C1%AF%DDX%E02%E3n%B2%EC%EB%B1%8B%A8%26%AC%00%BB%88%F3%20%FD%04%2C%8Dh%0C%3B%B2%97q%89HCx%94%19z%F0%E2%84%88%CF%C1%92%1C%2C%F4%B6%85%C8%10%DF8%5C%C9%14%FC%80%15%F9%A9%1E%C1%C1%3Ei%C7%CF%3F%23A%9B%3CE%0D%60%A1%A3%A4%3C%F7%B4%CF%A6%D32%B2%90%B2%C8%01%01%E2%E5%E76%05%E5%FD%B1%7D%0F%E6%1E%16%ECv%99%18c%D0J%F8%07%0D%A9%F8U%5E%5B%A1%95%28%B1%EB3%F2%EE%16%BC%D7%8F%E4%EC%C9uS%B6%83%B6%AEY%CF%2C%FF9%16%93%C9%5C%00%A8%16%00%8Fm%B1%A6%15%24z%2F%AE%CA%E6%92%28%E8%E7%17%83J%B6%15%D7%9E%9C%D7%AA%F1k%09%11%ED%86%97%2B%0B%EA%26%9E%F7%EF%C3.j0
1520295590.549592 TRACE: [ndn.security.v2.ValidationState] > Signature verification bypassed for data `/localhost/nfd/faces/update/h%06l%01%01p%01%01/%00%00%01a%F8%AD%BA%0F/%16%AE%29%A5%C1%BBZ%1E/%16%1C%1B%01%01%1C%17%07%15%08%04test%08%03KEY%08%08%D0%80%A8%A7Hr0%C6/%17%FD%01%00x%89%24%1C%3AK4%816%86%5B%B3%9Bh%F8%99DHU%F3%19A%3Eh%F9%BFLTX%92%9B%06%BE%9E%60%81%7B%B6%5D%96%F9%27%22e%88%88%40%EAr%EF%A1%20%E5%AA7Z%FC%C6%C1%AF%DDX%E02%E3n%B2%EC%EB%B1%8B%A8%26%AC%00%BB%88%F3%20%FD%04%2C%8Dh%0C%3B%B2%97q%89HCx%94%19z%F0%E2%84%88%CF%C1%92%1C%2C%F4%B6%85%C8%10%DF8%5C%C9%14%FC%80%15%F9%A9%1E%C1%C1%3Ei%C7%CF%3F%23A%9B%3CE%0D%60%A1%A3%A4%3C%F7%B4%CF%A6%D32%B2%90%B2%C8%01%01%E2%E5%E76%05%E5%FD%B1%7D%0F%E6%1E%16%ECv%99%18c%D0J%F8%07%0D%A9%F8U%5E%5B%A1%95%28%B1%EB3%F2%EE%16%BC%D7%8F%E4%EC%C9uS%B6%83%B6%AEY%CF%2C%FF9%16%93%C9%5C%00%A8%16%00%8Fm%B1%A6%15%24z%2F%AE%CA%E6%92%28%E8%E7%17%83J%B6%15%D7%9E%9C%D7%AA%F1k%09%11%ED%86%97%2B%0B%EA%26%9E%F7%EF%C3.j0`
1520295590.549854 DEBUG: [ndn.security.v2.ValidationState] > Internal implementation error (Validator/policy did not invoke success or failure callback)
Assertion failed: (!boost::logic::indeterminate(m_outcome)), fun1520295590.549944 TRACE: [ndn.security.v2.ValidationState] ~ValidationStatection ~ValidationState, file ../
src/security/v2/validation-state.cpp, line 44.
Abort trap: 6
</code></pre> NFD - Feature #4318: Content Store flush/erase commandhttps://redmine.named-data.net/issues/4318?journal_id=226182018-03-05T17:19:06ZJunxiao Shi
<ul><li><strong>Priority</strong> changed from <i>Urgent</i> to <i>Normal</i></li></ul><p>note-16 has nothing to do with this issue. Please open a separate bug with all informational requested in <a href="http://www.lists.cs.ucla.edu/pipermail/nfd-dev/2016-May/001748.html">http://www.lists.cs.ucla.edu/pipermail/nfd-dev/2016-May/001748.html</a></p>
NFD - Feature #4318: Content Store flush/erase commandhttps://redmine.named-data.net/issues/4318?journal_id=226192018-03-05T18:28:05ZAlex Afanasyev
<ul></ul><p>Junxiao, please be constructive and not obstructive. There is a problem that is seemingly caused by the commit related to this issue. I have provided all info to reproduce the problem and it was reliably reproduced by me, Davide, and Ashlesh. Help fix the problem, not instruct other on how to document the problem.</p>
NFD - Feature #4318: Content Store flush/erase commandhttps://redmine.named-data.net/issues/4318?journal_id=226202018-03-05T20:03:03ZAlex Afanasyev
<ul><li><strong>Priority</strong> changed from <i>Normal</i> to <i>High</i></li></ul><p>Ok. After spending a few hours. The issue directly relates to this issue: you managed to use the same TLV for two different fields in the ControlParameters: 135 code is used twice causing all type of trouble. Please renumber and submit patch asap, so we back to working NFD in the master branch.</p>
NFD - Feature #4318: Content Store flush/erase commandhttps://redmine.named-data.net/issues/4318?journal_id=226222018-03-05T23:11:59ZJunxiao Shi
<ul></ul><blockquote>
<p>you managed to use the same TLV for two different fields in the ControlParameters: 135 code is used twice causing all type of trouble.</p>
</blockquote>
<p>There is only one “135” in <a class="wiki-page" href="https://redmine.named-data.net/projects/nfd/wiki/ControlCommand">ControlCommand</a> r39. I don’t see a second assignment.</p>
NFD - Feature #4318: Content Store flush/erase commandhttps://redmine.named-data.net/issues/4318?journal_id=226232018-03-05T23:28:40ZDavide Pesavento
<ul></ul><p>Junxiao Shi wrote:</p>
<blockquote>
<blockquote>
<p>you managed to use the same TLV for two different fields in the ControlParameters: 135 code is used twice causing all type of trouble.</p>
</blockquote>
<p>There is only one “135” in <a class="wiki-page" href="https://redmine.named-data.net/projects/nfd/wiki/ControlCommand">ControlCommand</a> r39. I don’t see a second assignment.</p>
</blockquote>
<p><a href="https://github.com/named-data/ndn-cxx/blob/df5053824fc53eec6847067faa2da8b0ebb69917/src/encoding/tlv-nfd.hpp#L69">https://github.com/named-data/ndn-cxx/blob/df5053824fc53eec6847067faa2da8b0ebb69917/src/encoding/tlv-nfd.hpp#L69</a></p>
NFD - Feature #4318: Content Store flush/erase commandhttps://redmine.named-data.net/issues/4318?journal_id=226242018-03-06T00:21:52ZJunxiao Shi
<ul><li><strong>Related to</strong> <i><a class="issue tracker-1 status-5 priority-5 priority-highest closed" href="/issues/4538">Bug #4538</a>: Congestion mark related elements missing in ControlParameters</i> added</li></ul> NFD - Feature #4318: Content Store flush/erase commandhttps://redmine.named-data.net/issues/4318?journal_id=226282018-03-06T00:25:19ZJunxiao Shi
<ul><li><strong>Priority</strong> changed from <i>High</i> to <i>Normal</i></li></ul><blockquote>
<p><a href="https://github.com/named-data/ndn-cxx/blob/df5053824fc53eec6847067faa2da8b0ebb69917/src/encoding/tlv-nfd.hpp#L69">https://github.com/named-data/ndn-cxx/blob/df5053824fc53eec6847067faa2da8b0ebb69917/src/encoding/tlv-nfd.hpp#L69</a></p>
</blockquote>
<p>This looks like congestion control group's fault because they forgot to update <a class="wiki-page" href="https://redmine.named-data.net/projects/nfd/wiki/ControlCommand">ControlCommand</a>. Assigned to them in <a class="issue tracker-1 status-5 priority-5 priority-highest closed" title="Bug: Congestion mark related elements missing in ControlParameters (Closed)" href="https://redmine.named-data.net/issues/4538">#4538</a>.<br>
If I had reviewed ndn-cxx:commit:07d05c97b435a8de5ff76986df6a0cb3a50a6484, I would have checked the implementation matches protocol spec, and would not have approved the code without having the protocol in place first.</p>
NFD - Feature #4318: Content Store flush/erase commandhttps://redmine.named-data.net/issues/4318?journal_id=226302018-03-06T09:03:09ZDavide Pesavento
<ul></ul><p>Renumbering of <em>BaseCongestionMarkingInterval</em> is not going to happen. We have a full implementation based on it and we already shipped that code (and that TLV assignment) in v0.6.1. It is way simpler to just change the assignment for <em>NCsEntries</em>, since it was just merged and is still unimplemented in NFD.</p>
<p>And while at it, rename it to <em>NEntries</em> or <em>Count</em> like I said in note-4, since your previous argument no longer applies.</p>
NFD - Feature #4318: Content Store flush/erase commandhttps://redmine.named-data.net/issues/4318?journal_id=226432018-03-06T16:15:21ZDavide Pesavento
<ul></ul><p>Davide Pesavento wrote:</p>
<blockquote>
<p>And while at it, rename it to <em>NEntries</em> or <em>Count</em> like I said in note-4, since your previous argument no longer applies.</p>
</blockquote>
<p>Done in <a href="https://gerrit.named-data.net/4607">https://gerrit.named-data.net/4607</a></p>
NFD - Feature #4318: Content Store flush/erase commandhttps://redmine.named-data.net/issues/4318?journal_id=228722018-04-03T19:08:57ZJunxiao Shi
<ul><li><strong>% Done</strong> changed from <i>10</i> to <i>20</i></li></ul><p><a href="https://gerrit.named-data.net/4650">https://gerrit.named-data.net/4650</a> adds <code>Cs::erase</code> method. Like <code>Cs::find</code>, the API is asynchronous, to allow the possibility of offline (Solid State Drive) CS entries.</p>
NFD - Feature #4318: Content Store flush/erase commandhttps://redmine.named-data.net/issues/4318?journal_id=230142018-04-16T06:41:20ZJunxiao Shi
<ul><li><strong>% Done</strong> changed from <i>20</i> to <i>60</i></li></ul><p><a href="https://gerrit.named-data.net/4678">https://gerrit.named-data.net/4678</a> <code>CsManager</code></p>
NFD - Feature #4318: Content Store flush/erase commandhttps://redmine.named-data.net/issues/4318?journal_id=236332018-07-16T07:48:04ZJunxiao Shi
<ul></ul><p><a href="https://gerrit.named-data.net/#/c/NFD/+/4850">https://gerrit.named-data.net/#/c/NFD/+/4850</a> 'nfdc cs erase' command</p>
NFD - Feature #4318: Content Store flush/erase commandhttps://redmine.named-data.net/issues/4318?journal_id=236722018-07-19T07:53:03ZJunxiao Shi
<ul><li><strong>Status</strong> changed from <i>In Progress</i> to <i>Code review</i></li><li><strong>% Done</strong> changed from <i>60</i> to <i>100</i></li></ul> NFD - Feature #4318: Content Store flush/erase commandhttps://redmine.named-data.net/issues/4318?journal_id=236892018-07-23T01:22:20ZJunxiao Shi
<ul><li><strong>Status</strong> changed from <i>Code review</i> to <i>Closed</i></li></ul>