STEPHEN FARRELL ===============o Responses below only to things that aren't obviously right :) - the list of things a client must (not MUST?) do could really do with cross references to the relevant sections where a coder can find the stuff they need. OK. section 3.3: - I guess inverting the via list can fail if the topology has changed. Is this handled? What happens? If it's changed too much, the message just drops and you retransmit. I agree that this whole section is a mess. We'll see what we can do. Bruce: started this, but still needs more work - Why is it a SHOULD for the case where the 1st entry is on the connection table? Didn't you define the routing table just to make this distinction? If the SHOULD is right, then what's the exception? This should be a MUST, I believe. Cullen? Bruce? section 5.4.1: - Has anyone specified a topology plugin other than the MTI one from here? I'm wondering if the list here is really complete and doable. (It'd be easy to get this wrong.) Dunno. Bruce? Cullen? section 5.4.2.1: EKR: FIXED: - Which error SHOULD nodes sent if they cannot form connections? EKR: FIXED: EKR: FIXED: We'll figure this out. section 5.6.3.1: - Is the "no more aggressive than TFRC" sentence here clear? I thought it was, but perhaps not... Cullen, Bruce, want to weigh in? section 6.4.2.1: - I don't get the last sentence - how does the requester ask for the user's cert? And should that say owner's cert? (That may be just due to how long it's taken me to read this, at multiple sittings;-) There is a certificate kind (S 7). We'll add a forward ref. EKRTODO: This actually appears to be wrong. section 8: section 9.7: - You RECOMMEND that a peer maintain O(log(N)) things in the neighbour set. I'm not clear how the peer knows the value of N there? This seems like a real issue.... Bruce? Cullen? section 10.1 - This is not right! Why not use a real in the example? If the example could be verified that'd help coders. Sure, we can do that. - Using HTTP errors is fairly coarse grained. In particular don't you want to have a way to say "everything's fine but you asked for too many nodeids"? If there's a good HTTP error for that then calling it out would be good. Just a bare 4xx for username/password errors is fine though. We'll take a look. EKR: FIXED: section 12: EKR: FIXED: EKR: FIXED: - Do you need to reference the TLS re-negotiation bug RFC? Would it EKR: FIXED: have a bad impact here? (Not sure.) EKR: FIXED: EKR: FIXED: I think this is just part of the huge pile of things you need to get right with EKR: FIXED: TLS... section 13.5 - I don't get why you have two SIP entries there. It'd be nice to know. We'll add something. EKRTODO: Do something about this? The whole AppAttach/DTLS thing is very confusing. -Ekr ROBERT SPARKS ============= Page 38 - Section 5.2.1 - might be worth calling out that the sending node will be doing both e2e and hopbyhop retrasmissions Page 48 - Section 5.3.3 - should note 0x8000-0xfffe are reserved Page 82 - Section 6 - Can you find a different word than "unpartitioning" (or define it)? - it only occurs once in the document. Page 107 - the formula at the end of 9.5 has unbalanced parentheses Page 112 - In the implementer note in 9.7.4.3, can you point to references to get the implementer started? PETER SAINT-ANDRE ================= These make sense to fix. Few comments on some of them. 1. Regarding communication security, Section 1.3 states: Message Level: Each RELOAD message must be signed. Object Level: Stored objects must be signed by the creating peer. Did you mean "MUST" instead of "must"? We were trying to push the normative language to later .. we probably should have said "are" instead of "must be" but we can sort this out and fix it up one way or the other. 2. In Section 3.1.1, please provide references for TLS-PSK and TLS-SRP. 3. In Section 3.2.1, please provide a reference for TURN. 4. In Section 3.2.1, forward references would help for Attach (Section 5.1.1), Ping (5.5.3), Destination List (5.3.2.2), etc. 5. Section 3.2.2 mentions "the topology plugin required to act as a peer in the overlay"; does this assume a plugin architecture for Node implementations? Perhaps not, because in Section 5.4 the spec talks about a "Topology Plugin" in a more generic sense. It might be good to clarify. It means the generic sense ... makes sense to clarify. There no need for a plugin in the shared library install a plug in sense anywhere in the system. 6. The description of symmetric recursive routing in Section 3.3 makes me wonder whether messages themselves are stored at the nodes in the Via list; if so, does that introduce possible concerns related to privacy, security, and auditing? 7. Please expand "ICE" on first use (Section 1) and in Section 3.4. 8. Section 3.6.1 states that "the node does a DNS SRV lookup on the overlay name to get the address of a configuration server"; a forward reference to Section 10.2 would be appropriate here. Also, a reference to RFC 2782 seems in order here. 9. Section 3.6.2 states that "the enrollment server's ability to restrict attackers' access to certificates in the overlay is one of the cornerstones of RELOAD's security"; by "access" seems to be meant "privilege to be granted its own certificate", not "capacity to know the certificates of other nodes". 10. Section 4.1.1 states that "only a user with a certificate for 'alice@example.org' could write to that location in the overlay". At least a forward reference to Section 10.3 would help, which is the only place where the user name is a define as an rfc822Name (at least in the certificate). Furthermore, it would help to more clearly specify how a user name prepared and compared for authorization purposes. 11. In Section 5.1, what exactly does it mean for a peer to "pass a message up to the upper layers"? 12. Is there a reason why the end-to-end reliability mechanism in Section 5.2.1 doesn't recommend exponential backoff on retries? I think there is another thread on this one. 13. Section 5.3.2 states that "transaction IDs ... MUST be randomly generated"; perhaps a reference to RFC 4086 is in order? 14. In Section 5.3.3.1, it would be good to state whether the "error_info" text is intended to be presented to users (if so, we might need language tagging). It's not meant to go to a user. It's used an extension mechanism to pass bak things such as page 92 where we have If any type of request tries to access a data kind that the node does not know about, an Error_Unknown_Kind MUST be generated. The error_info in the Error_Response is: I think we need to provide some more information in 5.3.3.1 that clarifies what this is for and that it's not meant for end user error messages. 15. In Section 5.3.4, what exactly is "the identity used to form the signature"? It appears to be a Node-ID (or a hash thereof), but it would be good to make that clear in the definition (e.g., could it also be a Resource-ID)? 16. In Section 5.5.1.1, are "srflx", "prflx", and "relay" as defined for Candidate Types in ICE? The "type" is said to "correspond to the cand-type production" but it might help to point a bit more directly to RFC 5245. 17. Section 6.4.1.3 mentions previous versions of RELOAD. Given that no references are provided, is that mention helpful? I think this should be removed but would be interested to hear what my co-authors think on that. 18. The element is of type xsd:boolean; please note that this datatype has two lexical representations, "1" or "true" for the concept true and "0" or "false" for the concept false. You might want to point this out to implementers. The same applies to the other elements with a datatype of xsd:boolean (no-ice, clients-permitted, etc.). Good catch - I imagine the best thing to do here would be to mandate just one form. 19. Section 10.1 does not clearly specify which elements and attributes are required and which are optional. Is it assumed that the reader needs to refer to the schema for this information? Yes but no one object to changing that so the text indicated it as well as the schema but leave some small change of them getting out of sync as people extend the schema. Thoughts on what we should do ? I can go either way and I expect most the other people involved not to care. 20. In Section 12.3, it might be worthwhile to mention the possibility of attacks against the central enrollment authority. 21. In Section 13.6, it's still not clear what it means for an XML format to be binary-encoded. I think this needs to be more clearly specified, then the registration can reference the appropriate section of the spec. We should probably add a ref here. In this context we mean "binary" as defined in section 2.9 of RFC 2045. This data is definitely not 8bit. PETE RESNICK =========== ---------------------------------------------------------------------- COMMENT: ---------------------------------------------------------------------- 3.1 - The concept of "user" appears in this section along with "username", but it never really gets defined, or for that matter is understandable at all, until 6.3. Some introduction to "user" might be nice somewhere at or before 3.1. OK, we'll try to do something. As you know, thisi s inherently a bit amorphous. 5.1.1 - If a wildcard Node_ID is used, does the does the message get sent to the Message Transport for forwarding? Not spelled out in this section. 5.3.3.1 - Editing error: Error_Data_Too_Old is followed by Error_Data_Too_Old 5.3.4 - No discussion of wildcard in this section. Does there need to be? Good questions. We'll try to nail down the semantics here.. 5.5 - Not clear to me why this document settles on ICE (or anything lower layer)... Seems like it could have been abstracted out and left to a different layer. I'm kind of disappointed that all of section 5.5 (and maybe sections 5.6 and 8) is not a separate document. This was of course discussed but the consensus was taht it was better to have one long document than a bunch of documents you had to assemble into a suite. Obviously, opinions in IETF differ about what's best here... -Ekr STEWART BRYANT ============== == Bootstrap Node: A network node used by Joining Peers to help Nlocate the Admitting Peer. "Locate," rather than, "Nlocate?" == Routing Table: The set of peers which a node can use to route overlay messages. In general, these peers will all be on the connection table but not vice versa, because some peers will have Attached but not sent updates. I had to read this sentence several times to undersand it --maybe just be more explicit in terms of what could in which table while not being in the other table. ICE is used but not defined Finger table is used before it is described rather than defined, and the description really only becomes meaningful with knowledge of the CHORD paper. Skiplist could do with a reference other than requiring the reader to Google for the definition. > > > > > ---------------------------------------------------------------------- > COMMENT: > ---------------------------------------------------------------------- > > > I have a number of detailed comments/nits below. > > == > Bootstrap Node: A network node used by Joining Peers to help Nlocate > the Admitting Peer. > > "Locate," rather than, "Nlocate?" will fix > > == > > Routing Table: The set of peers which a node can use to route overlay > messages. In general, these peers will all be on the connection table > but not vice versa, because some peers will have Attached but not sent > updates. > > I had to read this sentence several times to undersand it --maybe just > be more explicit in terms of what could in which table while not being > in the other table. I think we can rewrite that to be much more comprehensible. It looks like something I wrote and I'm illiterate. We will have Bruce write it. > > ICE is used but not defined will fix > > Finger table is used before it is described rather than defined, and > the description really only becomes meaningful with knowledge of > the CHORD paper. will fix > > Skiplist could do with a reference other than requiring the reader > to Google for the definition. will fix > ADRIAN FARREL ============= ---------------------------------------------------------------------- COMMENT: ---------------------------------------------------------------------- DHT is used several times before it is expanded. --- Section 1.2 Topology Plugin: The Topology Plugin is responsible for implementing the specific overlay algorithm being used. It uses the Message Transport component to send and receive overlay management messages, to the Storage component to manage data replication, and directly to the Forwarding Layer to control hop-by-hop message forwarding. This component closely parallels conventional routing algorithms, but is more tightly coupled to the Forwarding Layer because there is no single "routing table" equivalent used by all overlay algorithms. Garbled English, I think. Try... Topology Plugin: The Topology Plugin is responsible for implementing the specific overlay algorithm being used. It uses the Message Transport component to send and receive overlay management messages, the Storage component to manage data replication, and the Forwarding Layer to control hop-by-hop message forwarding. This component closely parallels conventional routing algorithms, but is more tightly coupled to the Forwarding Layer because there is no single "routing table" equivalent used by all overlay algorithms. That is, the Topology Plugin does not use the Message Transport component to communicate with the Storage component (or if it does, your figure is broken). I also think I object to "closely parallels conventional routing algorithms". Unless I mistake it, the Topology Plugin has two functions: - constructing the local forwarding instructions - selecting the operational topology (i.e. creating links by sending overlay management messages). Personally, I think your work would be improved by separating topology management and forwarding determination within your architecture. --- The figure in Section 1.2 is missing "Usage Layer" and "Overlay Link Layer" both of which are described in the text. --- I think (somewhat understandably) your terminology keeps getting snarled. The problem is that if you reuse terms (like transport) in the overlay, then you can get pretty confused when you talk about the underlying infrastructure. You also have to get uncinfused about the use of the word "transport" in OSI model, and "transport" in the Thus... Overlay Link Layer: Responsible for actually transporting traffic directly between nodes. Each such protocol includes the appropriate provisions for per-hop framing or hop-by-hop ACKs required by unreliable transports. TLS [RFC5246] and DTLS [RFC4347] are the currently defined "link layer" protocols used by RELOAD for hop-by-hop communication. New protocols can be defined, as described in Section 5.6.1 and Section 10.1. As this document defines only TLS and DTLS, we use those terms throughout the remainder of the document with the understanding that some future specification may add new overlay link layers. The link layer doesn't transport traffic in the sense you have used "Message Transport". (It does do transort in t sense of a transport layered network, but I doubt you want to get into this confusion. Maybe s/transporting/delivering/ Additionally... "Each such protocol" - you have not mentioned protocols thus far. "hop-by-hop ACKs required by unreliable transports" - don't the ACKs (in the overlay link layer) make the link reliable? --- The figure at the end of 1.2 is really helpful. I wonder if you should change "real Internet" because (I think) your work is part of the Internet. I also think that your construct is arbitrarily nestable. It might be worth noting that you could run directly over a physical link using an appropriate encapsulation. That is, you do not need to run over calssic transport protocol spanning the Internet. --- The statement in 1.2.2 that The Message Transport component provides a generic message routing service for the overlay. is a highly confusing use of "routing". Is that what you meant to say? A reference for KBR would help. --- 1.2.5 This layer also utilizes a framing header to encapsulate messages as they are forwarding along each hop. s/forwarding/forwarded/ --- Section 2. Just for my clarity: "peers" are not necessarily adjacent in the overlay network? They are just the collection of nodes participating in an overlay instance? --- Section 2 Kind: A kind defines a particular type of data that can be stored in the overlay. Applications define new Kinds to store the data they use. Each Kind is identified with a unique integer called a Kind-ID. Please decide whether "Kind" or "kind" and aply to this paragraph and to the whole of the document. --- Section 2 Resource Name: The potentially human readable name Is "potentially human readable" helpful? --- Section 2 Are you sure that "Resource" and "Resource Name" are not circular definitions? Resource is an object associated with a string identifier. Resource Name is a name by which a resource is identified. But what is a resource in concrete terms? --- Section 2 There is some loose language around names and identifiers. Resource: An object or group of objects associated with a string identifier. See "Resource Name" Resource Name: The potentially human readable name by which a resource is identified. Resource-ID: A value that identifies some resources --- Section 2 Connection Table and Routing Table talk about Attach and Update way before they are discussed in the document. Would it be possible to phrase these definitions in slightly more abstract language? --- Section 2 Routing Table: The set of peers which a node can use to route overlay messages. Would it be helpful to state that the Routing Table contains "next hop peers"? Is this definition actually all there is to say about the Routing Table? It is just a list of peers that can be used to reach other non-connected peers? There is no information in the Routing Table about which peers can be reached through which non-connected peers? --- Section 2 Destination List: A list of IDs through which a message is to be routed. A single Node-ID is a trivial form of destination list. I think s/list of IDs/list of Node-IDs/ Can you clarify whether the list is complete? Is the list reduced hop by hop, or does it remain as a history? Does it include the source / node in hand? Does it include the destination? --- Section 2 Usage: A usage is an application that wishes to use the overlay for some purpose. Each application wishing to use the overlay defines a set of data kinds that it wishes to use. The SIP usage defines the location data kind. The word "application" causes ambiguity. The Microsoft Word running on my PC is an application instance. Microsoft Word is an application. "Word processors" is also an application (maybe Application Type?) --- Section 3.1 o To determine its position in the overlay topology when the overlay is structured. s/when/if/ ? --- Section 3.1 The general principle here is that the security mechanisms (TLS and message signatures) are always used, even if the certificates are self-signed. Is discussion of TLS in scope here? TLS would be in the data link layer in your architecture, wouldn't it? Or are you saying that TLS is used in the Message Transport component? --- Section 3.2 From the perspective of a peer, a client is simply a node which has not yet sent any Updates or Joins. Can you rephrase this in the abstract because we have not yet been told what an Update or a Join is? --- Section 3.2.1 forming a direct connections s/connections/conneciton/ --- Section 3.2.1 o Establish a connection with an arbitrary peer in the overlay (perhaps based on network proximity or an inability to establish a direct connection with the responsible peer). The term "responsible peer" has not been defined. I suspect you mean the term in the context of Section 1.1... Peers are responsible for storing the data associated with some set of addresses as determined by their Node-ID. --- Section 3.2.2 A node may act as a client simply because it does not have the resources or even an implementation of the topology plugin required to act as a peer in the overlay. "Resource" is a term with a very specific meaning in this document. You might want to use a different term here. --- 3.2.2 calculating the Resource-ID requires an implementation of the appropriate algorithm for the overlay. This discussion of an algorithm (and, indeed, calculating a Resource-ID) is novel. Isn't it enough to say that a client must sufficiently understand the nature of the overlay network to be able to determine the correct Resource-IDs to use? --- Section 3.3 Low state: RELOAD's routing algorithms must not require significant state to be stored on intermediate peers. "Significant" is subjective. What does this requirement mean? --- Section 3.3 The mechanism described as "iterative routing" is very fine. But the name is confusing. This is just route query, or route inspection? I guess that to operate the mechanism you iteratively query the hops along the path, in order to determine the explicit route: does that make it "iterative routing" or just an iterative query? --- Section 3.4 Say that peer A wishes to form a direct connection to peer B. The thing that triggers it to "wish" is interesting and might benefit from a cross-reference to the appropriate part of this I-D. I would note that In general, a peer needs to maintain connections to all of the peers near it in the Overlay Instance and to enough other peers to have efficient routing (the details depend on the specific overlay). is tautology. "Near" is a trocky word! You probably mean "near in the underlying Internet topology", but unless you are going to describe a way of knowing that (ALTO?) you can't leverage it. On the other hand, "near" in the overlay network means that there are connections (hence the tautology). Also: However, a peer should try to maintain the specified link set and if it detects that it has fewer direct connections, should form more as required. I don't find information about how that link set is specified. --- Section 5.1.2 If neither of the other three cases applies, then the peer MUST I think there are only two other cases. I.e. total of three for which this is the second. --- Section 5.1.2 and which is likely to be responsible for the first entry on the destination list I don't find that very helpful. In the fully connected networks, this may be likely. In the longer paths (such as in the examples in this document) or when Destination List is not a complete list, I don't think "likely" applies. --- Section 5.1.2 None of the above mechanisms are required for responses, since there is no need to ensure that subsequent requests follow the same path. Fixing the grammar and swapping the order to avoid some ambiguity... There is no requirement to ensure that a request issued after the receipt of a response follows the same path as the response. As a consequence, there is no reuirement to use either of the mechanisms described above (via list or state retention) when processing a response message. --- Section 5.3.2.1 I don't think it is helpful to mix and match hex and decimal values in this section. --- Section 5.4 As discussed in previous sections, RELOAD does not itself implement any overlay topology. Rather, it relies on Topology Plugins, which allow a variety of overlay algorithms to be used while maintaining the same RELOAD core. FWIW, I think RELOAD *does* implement the overlay topology. But it is the Topology Plugin that configures/determines the topology to be implemented. , I think RELOAD *does* implement the overlay topology. But it is the Topology Plugin that configures/determines the topology to be implemented. Eric Rescorla to Adrian, IESG, p2psip-chairs, draft-ietf-p2p. show details Sep 8 (4 days ago) Thanks for your review. > I am convinced that this document will prove to be extremely important > for the future of the Internet. That maybe makes the bar a little higher > and means that I would like to know the spec is really good. My baseline > is that the spec needs to reliably produce interoperable implementations > and, of course, work. This is our objective as well. > to efficiently route messages > = > > "Efficiency" in routing is a concept that may need qualifying. Maybe > you mean select an optimal path. But even that doesn't help much without > some explanation of the paradigm. As previously noted, I think this is fluff and would remove it. > Introduction > > High Performance Routing: The very nature of overlay algorithms > introduces a requirement that peers participating in the P2P > network route requests on behalf of other peers in the network. > This introduces a load on those other peers, in the form of > bandwidth and processing power. RELOAD has been defined with a > simple, lightweight forwarding header, thus minimizing the amount > of effort required by intermediate peers. > > Are you saying that that the look-up and forwarding has to be = > > lightweight? Or are you talking about the routing protocol? Or the > routing algorithm? = As is this, but in this case what it means is that it's inexpensive for a peer to take a packet and determine which of its adjacencies it should be routed to. > The concept of "overlay algorithm" is introduced in Section 1 without > really explaining it. Would you add a sentence to explain the purpose > of the algorithm. This will help shape the view of the rest of the > introduction and make it more readable. Sure. > A real problem with undertanding in this context is the distinction = > > between the algorithm that builds selects the nodes and links in the > overlay topology and that which decides the best path for any given = > > message. > > Reading the definition in section 2, the overlay algorithm is only > concerned with building the topology. So there must be a separate > algorithm hiding somewhere in the Topology Plugin that builds the > routing table. Well, the routing table in this case is just the list of adjacencies, and then there is some algorithm for selecting the optimal next hop given a particular routing table. However, in general the overlay construction algorithm and the optimal next hop are tightly related, and in RELOAD they are just specified together. > I do not believe that this document (and specifically section 9) = > > contains information to implement chord-reload without reference to > [chord]. That makes [chord] a normative reference. = > > > However, rather than that change, I would prefer this document to be > self-contained, and to include a full implementable description of = > > chord-reload. This is our intent, and what we thought we had done. Is there some particular thing you think isn't complete? > --- > > Section 3 gives some good input to management although I think the = > > section is misnamed because a large chunk of the text is about how > the network operates rather than how it is managed. > > I think you need to take a look at RFC 5706 for gidance on other = > > management concepts you should include. I am personally interested in > what OAM mechanisms you propose. (Yes, I do see section 5.5.3, but I = > > don't believe this is significant OAM in a layer network.) I think we're using management in the more general sense, namely how the network is constructed and stabilized. There's active work in the P2PSIP WG now on some more classic OAM-type stuff. > Section 3.3 > > Destination Lists: While in principle it is possible to just > inject a message into the overlay with a bare Node-ID as the > destination, RELOAD provides a source routing capability in the > form of "Destination Lists". A "Destination List provides a list > of the nodes through which a message must flow. > > s/bare/single/ ? > Spurious quote mark. > > Is this an ordered list? I assume so. Please say. > Is this a loose list or a strict list? I assume from the "single node" > option, this is loose. Please say. > What is the minimum content of a list? Presume the destination must be > there. Please say. We'll fix this. > Section 3.3 > > I am surprised that the sending-hop node is added to the Via List (in > the figure) by the receiving-hop node. This is not optimal and might be > "entertaining" on multi-access links if such could ever exist. > > I would have expected that the sending-hop would add itself prior to = > > sending the message. This is what other technologies do. Why isn't it optimal? I don't think that the multi-access link question is relavant here, since all the data is sent over point-to-point secured links, so there's never any confusion. > Section 3.3 > > > The second figure (for truncated via lists) is broken or confused. Agreed. We'll fix. > Section 3.5.1 > > I am having trouble understanding how all nodes in an Overlay Instance > arrange to use the same Overlay Algorithm (which I believe is a = > > requirement). = > > Maybe there is a parameter on a Join? > Could this section explain? All nodes in the overlay have the same algorithm. We'lll see what we can do about clearing that up. > Section 3.6.1 > > Should the figure in 1.2 show a component for "Bootstrap and > Configuration"? My issue here is that DNS lives below the Overlay Link = > > Service Boundary line. Similarly, HTTP is below the line. In some ways > the uses of these services is analagous to the use of ARP and DHCP in > bottstrapping routers and hosts. Can you show how these services fit > in on the diagrams in 1.2? I'm not sure that's actually going to be helpful. This diagram is pretty crowded already and we're stressing the analogy pretty hard as it is. > Section 5.1.1 > > If the message is a response and there is state for > the transaction ID, the state is reinserted into the destination > list before the message is further processed. > > There is too much passive voice here for me to work out who carries out > this action. Yes, we'll try to rewrite. > Section 5.2 > > An overlay may be configured to use alternative routing > algorithms, and alternative routing algorithms may be selected on a > per-message basis. > > s/may/MAY/ twice > > I think there is something missing in the description. Is the per- > message selection based on a parameter on the message or a local policy? > Does the same routing algorithm need to be used end-to-end for any one > message? (If so, how achieved if not being signaled on the message?) We'll try to rewrite. > Section 5.2.1 > > Parallel searches for the resource are a common solution to improve > reliability in the face of churn or of subversive peers. = > > > The concept of a "search" is ambiguous here. you have just mentioned = > > doing an enquiry to the Topology Plugin to determine the appropriate > next hop. Thus, it is not clear whether the search is an algorithm for > searching the routing table, or an algorithm for probing the network. > In the latter case, it is unclear whether the probe is using the message > itself, or using the iterative route query mechanism. Agreed. We'll adjust. > Section 5.3.2 > > version: The version of the RELOAD protocol being used. This is a > fixed point integer between 0.1 and 25.4. This document describes > version 0.1, with a value of 0x01. [[ Note to RFC Editor: Please > update this to version 1.0 with value of 0x0a and remove this > note. ]] > > Can you explain why this document has a value in the version field other > than the value that will be in the RFC? Obviously no-one has implemented > and deployed a pre-standard version. I'm not sure what the question is here. Certainly, people have deployed pre-standard versions of this protocol. > Section 5.3.2 > > ttl: An 8 bit field indicating the number of iterations, or hops, a > message can experience before it is discarded. The TTL value MUST > be decremented by one at every hop along the route the message > traverses. If the TTL is 0, the message MUST NOT be propagated > further and MUST be discarded, and a "Error_TTL_Exceeded" error > should be generated. The initial value of the TTL SHOULD be 100 > unless defined otherwise by the overlay configuration. > > I think there is a little ambiguity about the moment that TTL is > decremented. Could be decrement on tx or rx. Makes a difference to > when a message is discarded, etc. Yes. We'll come to a conclusion. > Section 5.3.2.3 > > If the > RESPONSE_COPY flag is set, any node generating a response MUST > copy the option from the request to the response except that the > RESPONSE_COPY, FORWARD_CRITICAL and DESTINATION_CRITICAL flags > must be cleared. > > s/must/MUST/ OK. > Section 5.4.2.1 > > In general, nodes which cannot form connections SHOULD report an > error. However, implementations MUST provide some mechanism whereby > nodes can determine that they are potentially the first node and take > responsibility for the overlay. This specification does not mandate > any particular mechanism, but a configuration flag or setting seems > appropriate. > > Isn't it the case that by definition a node that cannot form its first > connection is the first node. That is, it is perfectly possible for a > network to become partitioned and for parts of the network to come up > and operate on their own. It does not seem to me that a configuration > option can instruct a node that it is or is not the first node. No, it can tell it that it's potentially a bootstrap node. If it knows that the bootstrap nodes are up, then if you can't talk to it then you're likely partitioned. > Section 5.6.3.1.1 > > A node SHOULD retransmit a message if it has not received an ACK > after an interval of RTO ("Retransmission TimeOut"). The node MUST > double the time to wait after each retransmission. In each > retransmission, the sequence number is incremented. > > Please clarify that this is the message originator, not a transit node. Willdo. > Section 9.3. > > If a peer is not responsible for a Resource-ID k, but is directly > connected to a node with Node-ID k, then it routes the message to > that node. Otherwise, it routes the request to the peer in the > routing table that has the largest Node-ID that is in the interval > between the peer and k. If no such node is found, it finds the > smallest Node-Id that is greater than k and routes the message to > that node. > > Is it not possible that no such node is found, but the network is still > connected. > > m---n---k > > Trying to reach k > > n < m < k > > k is not directly connected to m > n is not in the interval m to k > n is not greater than k Comparisons are done mod-2^128. We'll add clarifying text. Responses to the comments below which ren't no-brainers. > Section 1.2 > > Topology Plugin: The Topology Plugin is responsible for implementing > the specific overlay algorithm being used. It uses the Message > Transport component to send and receive overlay management > messages, to the Storage component to manage data replication, and > directly to the Forwarding Layer to control hop-by-hop message > forwarding. This component closely parallels conventional routing > algorithms, but is more tightly coupled to the Forwarding Layer > because there is no single "routing table" equivalent used by all > overlay algorithms. > > Garbled English, I think. Try... > > Topology Plugin: The Topology Plugin is responsible for implementing > the specific overlay algorithm being used. It uses the Message > Transport component to send and receive overlay management > messages, the Storage component to manage data replication, and > the Forwarding Layer to control hop-by-hop message forwarding. = > > This component closely parallels conventional routing algorithms, > but is more tightly coupled to the Forwarding Layer because there > is no single "routing table" equivalent used by all overlay = > > algorithms. > > That is, the Topology Plugin does not use the Message Transport = > > component to communicate with the Storage component (or if it does, your > figure is broken). Thanks. > I also think I object to "closely parallels conventional routing > algorithms". Unless I mistake it, the Topology Plugin has two functions: > - constructing the local forwarding instructions > - selecting the operational topology (i.e. creating links by sending > overlay management messages). We'll remove that phrase. > Personally, I think your work would be improved by separating topology > management and forwarding determination within your architecture. Well, they're tightly related. > The figure in Section 1.2 is missing "Usage Layer" and "Overlay Link = > > Layer" both of which are described in the text. > > --- > > I think (somewhat understandably) your terminology keeps getting > snarled. The problem is that if you reuse terms (like transport) in > the overlay, then you can get pretty confused when you talk about > the underlying infrastructure. You also have to get uncinfused about > the use of the word "transport" in OSI model, and "transport" in the > > Thus... > > Overlay Link Layer: Responsible for actually transporting traffic > directly between nodes. Each such protocol includes the > appropriate provisions for per-hop framing or hop-by-hop ACKs > required by unreliable transports. TLS [RFC5246] and DTLS > [RFC4347] are the currently defined "link layer" protocols used by > RELOAD for hop-by-hop communication. New protocols can be > defined, as described in Section 5.6.1 and Section 10.1. As this > document defines only TLS and DTLS, we use those terms throughout > the remainder of the document with the understanding that some > future specification may add new overlay link layers. > > The link layer doesn't transport traffic in the sense you have used > "Message Transport". (It does do transort in t sense of a transport = > > layered network, but I doubt you want to get into this confusion. > > Maybe s/transporting/delivering/ I'm happy to change transporting to delivering here. > Section 2. > > Just for my clarity: "peers" are not necessarily adjacent in the overlay > network? They are just the collection of nodes participating in an > overlay instance? = Yes. > Section 2 > > Resource Name: The potentially human readable name > > Is "potentially human readable" helpful? Probably not. > Section 2 > > Are you sure that "Resource" and "Resource Name" are not circular > definitions? > > Resource is an object associated with a string identifier. = > > Resource Name is a name by which a resource is identified. > > But what is a resource in concrete terms? I'm not sure I understnad this question. This is a hash table, so there are objects and they have names that point to them. > Section 2 > > There is some loose language around names and identifiers. > > Resource: An object or group of objects associated with a string > identifier. See "Resource Name" > > Resource Name: The potentially human readable name by which a > resource is identified. > > Resource-ID: A value that identifies some resources I'm still not sure what the problem is here. > Section 2 > > Routing Table: The set of peers which a node can use to route > overlay messages. > > Would it be helpful to state that the Routing Table contains "next hop > peers"? > > Is this definition actually all there is to say about the Routing Table? > It is just a list of peers that can be used to reach other non-connected > peers? There is no information in the Routing Table about which peers > can be reached through which non-connected peers? Correct. Because the topology is largely determined by the coordinate system, you don't need this. > Section 2 > > Destination List: A list of IDs through which a message is to be > routed. A single Node-ID is a trivial form of destination list. > > I think s/list of IDs/list of Node-IDs/ > > Can you clarify whether the list is complete? Is the list reduced hop > by hop, or does it remain as a history? Does it include the source / = > > node in hand? Does it include the destination? We'll clarify. > Section 2 > > Usage: A usage is an application that wishes to use the overlay for > some purpose. Each application wishing to use the overlay defines > a set of data kinds that it wishes to use. The SIP usage defines > the location data kind. > > The word "application" causes ambiguity. > The Microsoft Word running on my PC is an application instance. = > > Microsoft Word is an application. = > > "Word processors" is also an application (maybe Application Type?) Yeah, we weren't sure what to do about that, but we're short on words... > Section 3.2 > > From the perspective of a peer, a client is simply a node > which has not yet sent any Updates or Joins. > > Can you rephrase this in the abstract because we have not yet been told > what an Update or a Join is? Yes. > Section 3.2.2 > > A node may act as a client simply because it does not have the > resources or even an implementation of the topology plugin required > to act as a peer in the overlay. > > "Resource" is a term with a very specific meaning in this document. You = > > might want to use a different term here. Fair enough. > --- > > Section 3.3 > > Low state: RELOAD's routing algorithms must not require > significant state to be stored on intermediate peers. > > "Significant" is subjective. What does this requirement mean? It's fluff... > Section 3.3 > = > > The mechanism described as "iterative routing" is very fine. But the > name is confusing. This is just route query, or route inspection? > I guess that to operate the mechanism you iteratively query the hops > along the path, in order to determine the explicit route: does that > make it "iterative routing" or just an iterative query? This sounds like a semantic question too complicated for me... Bruce? Cullen? > Section 3.4 > > Say that peer A wishes to form a direct connection to peer B. > > The thing that triggers it to "wish" is interesting and might benefit > from a cross-reference to the appropriate part of this I-D. > > I would note that > In general, a peer needs to maintain connections to all of the peers > near it in the Overlay Instance and to enough other peers to have > efficient routing (the details depend on the specific overlay). > is tautology. "Near" is a trocky word! You probably mean "near in the > underlying Internet topology", but unless you are going to describe a > way of knowing that (ALTO?) you can't leverage it. On the other hand, > "near" in the overlay network means that there are connections (hence > the tautology). Actually, in this case "near" means nearby in the overlay coordinate system (in Chord, that means numerically close to). > Also: > However, a peer should try to maintain the > specified link set and if it detects that it has fewer direct > connections, should form more as required. > I don't find information about how that link set is specified. It's overlay specific. We'll add a forward refrence. -Ekr Hi Eric, Very many thanks for engaging so constructively on my rambling Discuss and Comments. I have cut down to just the points where there is valuable information exchange left to be had. > > "Efficiency" in routing is a concept that may need qualifying. Maybe > > you mean select an optimal path. But even that doesn't help much without > > some explanation of the paradigm. > > As previously noted, I think this is fluff and would remove it. OK > > A real problem with undertanding in this context is the distinction > > between the algorithm that builds selects the nodes and links in the > > overlay topology and that which decides the best path for any given > > message. > > > > Reading the definition in section 2, the overlay algorithm is only > > concerned with building the topology. So there must be a separate > > algorithm hiding somewhere in the Topology Plugin that builds the > > routing table. > > Well, the routing table in this case is just the list of adjacencies, > and then there is some algorithm for selecting the optimal next > hop given a particular routing table. However, in general the > overlay construction algorithm and the optimal next hop > are tightly related, and in RELOAD they are just specified > together. Yes, I get all that. It just took some getting! I think that, knowing how topology creation and routing work, I was looking for specific abstract functions. It was easy to tell that the intention was that this was all in the Topology Plugin, but not easy to distinguish the different components. They're all there: it was just fuzzy. > > I do not believe that this document (and specifically section 9) > > contains information to implement chord-reload without reference to > > [chord]. That makes [chord] a normative reference. > > > > However, rather than that change, I would prefer this document to be > > self-contained, and to include a full implementable description of > > chord-reload. > > This is our intent, and what we thought we had done. Is there some > particular thing you think isn't complete? Well issues include: - Section 9 starts by listing the differences from [chord]. This can't be useful unless I have read [chord]. Maybe move to 9.10 and note that it is provided for information only? - What is a finger table? Maybe rewrite this section using terminology from this document and not terminology from [chord]? - Ditto neighbor table - What is a DHT ring? - Does it matter to the results of the various algorithms how the data is stored? The efficiency of lookup is very interesting, but that is technically an implementation detail. > > Section 3 gives some good input to management although I think the > > section is misnamed because a large chunk of the text is about how > > the network operates rather than how it is managed. > > > > I think you need to take a look at RFC 5706 for guidance on other > > management concepts you should include. I am personally interested in > > what OAM mechanisms you propose. (Yes, I do see section 5.5.3, but I > > don't believe this is significant OAM in a layer network.) > > I think we're using management in the more general sense, namely > how the network is constructed and stabilized. There's active > work in the P2PSIP WG now on some more classic OAM-type > stuff. Well, forward pointers to other I-Ds might help a lot. In general, however, if you are building such a comprehensive new architecture, I would ask you to include the architectural description of OAM in this document and only punt the protocol details to other documents. > > Section 3.3 > > > > I am surprised that the sending-hop node is added to the Via List (in > > the figure) by the receiving-hop node. This is not optimal and might be > > "entertaining" on multi-access links if such could ever exist. > > > > I would have expected that the sending-hop would add itself prior to > > sending the message. This is what other technologies do. > > Why isn't it optimal? I don't think that the multi-access link question > is relavant here, since all the data is sent over point-to-point > secured links, so there's never any confusion. In this mode, the receiver must process each message in the context of the link on which it arrived. Of course this is possible, but it complicates an implementation. The sender knows its own identity and can add it easily. The receiver has to take the information about the incoming link and look up the identity of the remote link end (not a huge burden, but...) Who is to say that P2P links today will not become something more exotic tomorrow? > > Section 3.5.1 > > > > I am having trouble understanding how all nodes in an Overlay Instance > > arrange to use the same Overlay Algorithm (which I believe is a > > requirement). > > > > Maybe there is a parameter on a Join? > > Could this section explain? > > All nodes in the overlay have the same algorithm. We'lll see what we > can do about clearing that up. Hooray, I understood correctly! It is the coordination of this fact (which might suffer from config errors) that is interesting. > > Section 3.6.1 > > > > Should the figure in 1.2 show a component for "Bootstrap and > > Configuration"? My issue here is that DNS lives below the Overlay Link > > Service Boundary line. Similarly, HTTP is below the line. In some ways > > the uses of these services is analagous to the use of ARP and DHCP in > > bottstrapping routers and hosts. Can you show how these services fit > > in on the diagrams in 1.2? > > I'm not sure that's actually going to be helpful. This diagram > is pretty crowded already and we're stressing the analogy pretty > hard as it is. Hmmm, well. I had to read as far as 3.6.1 to discover that the two figures in 1.2 are missing an interface to some underlying services, and that there is a component in the overlay responsible for accessing those services. > > Section 5.3.2 > > > > version: The version of the RELOAD protocol being used. This is a > > fixed point integer between 0.1 and 25.4. This document describes > > version 0.1, with a value of 0x01. [[ Note to RFC Editor: Please > > update this to version 1.0 with value of 0x0a and remove this > > note. ]] > > > > Can you explain why this document has a value in the version field other > > than the value that will be in the RFC? Obviously no-one has implemented > > and deployed a pre-standard version. > > I'm not sure what the question is here. Certainly, people have > deployed pre-standard versions of this protocol. I am happy to know that pre-standard implementations exist and that this has fed back to this development. But I don't think we (the IETF) are enthusiastic about recognizing pre-standard implementation and deployments as granting them de facto existence in the standards set by allocating codepoints. In particular, you have assigned a codepoint in a draft that allows people to implement the draft yet there is the normal pre-amble in the draft boilerplate that explains why this would be ill-advised. How about this as a suggestion... version: The version of the RELOAD protocol being used. This is a fixed point integer between 1.0 and 25.4. This document describes version 1.0 with value of 0x0a. Messages carrying unrecognized version values MUST be > > Section 5.4.2.1 > > > > In general, nodes which cannot form connections SHOULD report an > > error. However, implementations MUST provide some mechanism whereby > > nodes can determine that they are potentially the first node and take > > responsibility for the overlay. This specification does not mandate > > any particular mechanism, but a configuration flag or setting seems > > appropriate. > > > > Isn't it the case that by definition a node that cannot form its first > > connection is the first node. That is, it is perfectly possible for a > > network to become partitioned and for parts of the network to come up > > and operate on their own. It does not seem to me that a configuration > > option can instruct a node that it is or is not the first node. > > No, it can tell it that it's potentially a bootstrap node. If it knows > that the bootstrap nodes are up, then if you can't talk to it then > you're likely partitioned. OK, I think. In this case, the problem is with "some mechanism whereby nodes can determine that they are potentially the first node". I think the word "potentially" is what threw me. > > Section 9.3. > > > > If a peer is not responsible for a Resource-ID k, but is directly > > connected to a node with Node-ID k, then it routes the message to > > that node. Otherwise, it routes the request to the peer in the > > routing table that has the largest Node-ID that is in the interval > > between the peer and k. If no such node is found, it finds the > > smallest Node-Id that is greater than k and routes the message to > > that node. > > > > Is it not possible that no such node is found, but the network is still > > connected. > > > > m---n---k > > > > Trying to reach k > > > > n < m < k > > > > k is not directly connected to m > > n is not in the interval m to k > > n is not greater than k > > Comparisons are done mod-2^128. We'll add clarifying text. Yeah, I just found this on re-reading and before I got to your response. There is a casual throw-away line at the end of Section 9.1 that I missed or thought applied only to the Resource-ID responsibility work. Adding the same note in 9.3 would be cool. > > Personally, I think your work would be improved by separating topology > > management and forwarding determination within your architecture. > > Well, they're tightly related. Yes. I do understand that. And the chord-reload relationship is particularly tight. But, one could separate them very thoroughly and still have a functioning RELOAD system. For example, topology creation could be configured and manual, while routing discovery could be achieved entirely using the send-update option on RouteQuery. This was a Comment, and I am not pushing hard. Just what I thought. > > Section 2 > > > > Are you sure that "Resource" and "Resource Name" are not circular > > definitions? > > > > Resource is an object associated with a string identifier. > > Resource Name is a name by which a resource is identified. > > > > But what is a resource in concrete terms? > > I'm not sure I understnad this question. This is a hash table, so > there are objects and they have names that point to them. Oh, where is Wittgenstein when you need him? So I understand that a Resource is nothing more than some thing (== object) that is identified by a Resource Name. And so the very generality makes the pair of definitions circular. Is a data file a Resource before I assign it a Resource Name? I guess not. OK. This is probably all natural and obvious to people who live in Applicationland. Let's drop the issue. > > Section 2 > > > > There is some loose language around names and identifiers. > > > > Resource: An object or group of objects associated with a string > > identifier. See "Resource Name" > > > > Resource Name: The potentially human readable name by which a > > resource is identified. > > > > Resource-ID: A value that identifies some resources > > I'm still not sure what the problem is here. What is the difference between a Resource Name and a Resource-ID? Possibly this difference is hiding in the contrast of "string identifier" and "value". The definitions do seem to overlap because Resource Name is only "potentially human readable" while a Resource-ID could also be human readable because "often this is not human friendly/readable". Re-reading (I have nothing else to do at Beijing airport this evening) I finally narrowed it down. Resource-ID contains a good statement on its purpose. Resource Name does not - we suffer from the dreaded passive voice... The potentially human readable name by which a resource is identified. Identified to whom, by whom, in what context? > > Section 2 > > > > Usage: A usage is an application that wishes to use the overlay for > > some purpose. Each application wishing to use the overlay defines > > a set of data kinds that it wishes to use. The SIP usage defines > > the location data kind. > > > > The word "application" causes ambiguity. > > The Microsoft Word running on my PC is an application instance. > > Microsoft Word is an application. > > "Word processors" is also an application (maybe Application Type?) > > Yeah, we weren't sure what to do about that, but we're short on > words... "Class of applications"? "Application type"? Or let it go because you Applheads know what you mean. (Was William Tell an Applhead?) > > Section 3.3 > > > > Low state: RELOAD's routing algorithms must not require > > significant state to be stored on intermediate peers. > > > > "Significant" is subjective. What does this requirement mean? > > It's fluff... :-) I wonder if you need to create Section 16. Fluff You can collect it all into one section for ease of reading by the PLM people. > > Section 3.3 > > > > The mechanism described as "iterative routing" is very fine. But the > > name is confusing. This is just route query, or route inspection? > > I guess that to operate the mechanism you iteratively query the hops > > along the path, in order to determine the explicit route: does that > > make it "iterative routing" or just an iterative query? > > This sounds like a semantic question too complicated for me... > Bruce? Cullen? Oh, who cares? I was reviewing rather late at night and trying to digest the whole of your little document. > > Section 3.4 > > > > I would note that > > In general, a peer needs to maintain connections to all of the peers > > near it in the Overlay Instance and to enough other peers to have > > efficient routing (the details depend on the specific overlay). > > is tautology. "Near" is a tricky word! You probably mean "near in the > > underlying Internet topology", but unless you are going to describe a > > way of knowing that (ALTO?) you can't leverage it. On the other hand, > > "near" in the overlay network means that there are connections (hence > > the tautology). > > Actually, in this case "near" means nearby in the overlay coordinate > system (in Chord, that means numerically close to). Aha! Good. Suggest you continue to use the word "near" but add something like (modulo making it correct :-) "In the context of RELOAD, 'near' means close-by in the topology medium in which the overlay operates. For example, in the context of chord-reload (see Section 9), 'near' means numerically close in the coordinate system that the topology algorithm uses." Cheers, Adrian