Reps as Gateways Network Topology

For the purposes of reducing bandwidth and generally ensuring that the PR's can concentrate their effort on efficiently arriving at consensus.

I couldn't find much documentation on the existing peering process but I'm under the impression that currently every node connects and broadcasts to every other node. Instead, we could potentially leverage the voting Representatives (non-PR's) as the gatekeepers to the internal consensus network.

  • PR's operate a fully connected mesh only with other PR's
  • PR's only accept peer connections from voting nodes (not Edge Nodes)
  • Non-PR Reps maintain peered connections to only a few PR's (random or random+telemetry)
  • Reps act as gateways and relay transactions from Edge nodes
  • Edge nodes maintain peer connections to a few Reps only (random or random+telemetry)
  • Reps only send cemented blocks back to Edge nodes
  • Edge nodes could be sparsely populated and drop blocks for accounts they have no interest in (orthogonal to bandwidth issue)

If enforcement was required it might involve a period tally vote on which edge nodes are connected and a ban on Edge nodes that are exceeding quota.

Apologies if this kind of thing has been discussed previously. I couldn't find anything obvious.

7 Likes

How does a voting Representative become a Primary Representative? Or were you thinking of the current Principal Representative with vote weight above some threshold (currently set to 0.1% of all online vote weight)

So no changes to how you become a PR. My understanding is that is done on delegated vote and wouldn't need to change. The only thing that would happen is at the point that a regular Rep got promoted to become a PR then the network might need to do a bit of re-organisation. That new PR would disconnect any Edge nodes and force them to find another Rep etc.

1 Like

The network topology is definitely worth digging in to for improvement. You're right the current topology is fully-connected, it definitely needs to be converted to a mesh network. Last time I looked Kademlia was the state of the art DHT structure.

I think there's room for differentiating peers by a category e.g. PR, rep, historical node, routing node? Then the node hardware could be picked for a specific purpose.

4 Likes

Hey, this is an interesting idea. It's the first proposal I've read that discusses the network topology of Nano.

I have some questions. How does a Node become a "Representative"? Would it become an issue if a "Representative" node becomes a bad actor and prevent certain edge nodes from connecting? Also, could banning edge nodes be abused to censor groups/or transactions?
How did you come to choose 3 connected peers?
Also, is it possible that 3 PR representatives connected to a "Representative" can collude and send an incorrect block?

Suggestion:
Could this network include a non-voting PR node (Bootstrapping Node) that is in charge of bootstrapping nodes that are falling behind? They watch confirmed nodes and update their ledger. It becomes trusted via being voted in by the voting PR nodes. PR nodes must choose a Bootstrapping Node and must be chosen by at least X amount of voting weight from PRs (let's say 10%).

So my understanding is that currently, any node can become a voting Representative by having a minimum level of delegated stake in its wallet (min 1000 Nano I believe) and switching on voting mode. That doesn't have to change.

If clients (e.g Edge connecting to Rep or Rep connecting to PR) choose 3 random nodes from the set of available nodes then they should be getting the same data stream from 3 nodes and it makes it possible to detect a bad actor (or a broken node) if one of those streams is out of sync with the other two - at which point they can replace that faulty node with another chosen at random.

Further catagorisation of nodes may be desirable in the future but I thought an easy first step was to assume that all Reps are also Historic nodes that can be used for bootstrapping. So a new Rep would connect to 3 other Reps (Not PR's) for bootstrapping and could pull the full account chain from one Rep node and then verify the frontier block via the other 2 Reps to remove the possibility of bootstrap poisoning.

Likewise, sparsely populated Edge nodes could use the same bootstrapping trick but on the fly when they become aware of a new account that they need to interact with. This would mean that Edge nodes start off with an empty ledger and that's fine. Whenever a transaction is sent through it, then if that is for an account not already in the ledger then the Edge node can pull the frontier block from the 3 rep nodes and keep a pruned ledger for only the accounts that it cares about. This would massively reduce the overhead of running an Edge node.

This is a nice overview of Kadcast (kademlia broadcast)

Might be a good idea to simulate different schemes in order to understand what might happen to confirmation latency.

A good read. It mentions Kadcast security depends on using a POW nonce during the node ID creation. I'm guessing Nano wouldn't need this since ORV already ensures a majority of trustful nodes. If I understand correctly then security could also be increased the same way they suggest handling reliability - by picking more delegates per bucket.

So would only the PRs use Kadcast? If so how would reps and edge nodes connect?