Dynamically adjust minimum work difficulty as a function of network utilization

Had a bit of thinking of how nano should convey its message to the world and I don't think being a free-to-use network is the best one. The main reason for why that is bad is because it's intercepted as spam sensitive, and for good reasons. Data storage and bandwidth cost money, even if it is an extremely efficient protocol like Nano. That is why PoW was introduced as the first layer of resistance, also why the PoW threshold now is increased, for good and bad. Nano is NOT a free-to-use network. Pay the PoW, whatever it is, and build the services to handle it. If users have to wait for PoW, then so be it. It's not that different from fee-based cryptocurrencies in the end. It's all about introducing a cost of using the network, and Nano doing it a bit different but with many advantages. I think the static PoW change is fair while we are waiting for even more advanced DDOS protection to be built-in as a second layer of protection. Sure I also would like Nano to be free/effortless to use, with ultra-fast PoW for everyone but that's not the reality. We have to accept the fact that if Nano is going to be scaled significantly, so will the PoW, wherever it's in the protocol or not. If Nano gains serious traction, this WILL happen, either from evil actors trying to shut it down OR from genuine use. It's also not that difficult to solve increased work cost for transaction providers (wallets, exchanges, etc) using pooled PoW services and perhaps offer premium features to pay for it.

Now, how about this proposal as another layer of protection.
The dynamic PoW that was introduced to allow transactions with a higher level get prioritized was a good solution but it's not perfect. For this to happen the network has to first be saturated and at that point, the network does no longer operate under optimal circumstances. For example, the confirmation speed is slower compared to when slightly below the saturation level. What if the required minimum PoW would increase as a function of network load, and drop blocks that don't follow the rule? So as an example if cps is 10 the pow become 2x, 20 -> 4x, 40 -> 8x. This method would invalidate pre-computed chunks of spam blocks and it would be really expensive to live spam. Some regular pow could become invalid as well, like in wallets but maybe it's worth it? They can always re-compute. I'm thinking this minimum PoW level could be based as an average of last 1h CPS performance (or longer). Then pre-computed work won't break because of peaks in network load, but would be during prolonged stress or general increase of network utilization. In the end, the network "use cost" would self-balance as a function of ledger data storage rate and node bandwidth.

4 Likes

i was thinking the same thing, nano needs dynamic difficulty to prevent spam.

Easy way to do that is :
Difficulty = 8 * ( 15 min TPS )

It sounds like you're looking for a way nodes can throttle activity a bit to keep from going too far into the saturation point, which causes a drop in confirmation rates. It is a tricky one, and my initial reaction is against the idea of adjusting difficulty to cause more aggressive dropping of blocks. It seems like other, less blunt methods may be available to allow nodes to better manage staying near maximum throughput, with dropping as a last resort.

I don't believe we have a clear enough picture of the cause of the confirmation slowdown at saturation, so we will likely need to discover more details across various situations. Is it block propagation? Block processing? Vote processing? This likely varies by node and type of traffic being seen, but perhaps some of these scenarios become identifiable as they build up.

One thing I've seen discussed is using back pressure to better manage activity. The internals of the node are complex and mostly beyond my grasp here, so this may be speaking beyond my expertise, but finding computationally light metrics to monitor and dynamically throttling the types of activities that would escalate issues when those metrics are hitting peak, may be worth exploring.

Tracking confirmation rates vs. backlog of elections may be the top level indicator that adjustments need to be made when that is peaking. Once a potential ceiling is seen, looking at the shape of voting traffic vs. block traffic or other details to determine where some activities might be slowed down, queued for later, etc. could allow some self-healing of sorts.

And of course this exploration could lead us right back to realizing that because of block processing or other reasons we did indeed need to escalate difficulty and drop blocks as you suggest. Figuring this out will be a little complicated, so any help sorting it out is welcome :sweat_smile:

This is an interesting video covering some of these capacity management ideas related to Little's Law, might be worth a watch. https://www.youtube.com/watch?v=m64SWl9bfvk

3 Likes

Thanks for your explanation! It’s way beyond my tech skill all of this really. Just trying to get a feeling for the root fundamentals of what Nano is trying to achieve long term.

:+1: I hope someone comes along and says I'm thinking about this wrong and it is much simpler :smile: - @Dotcom or @Srayman perhaps?

I think you summarized it well. The one big thing I would add is identifying ways to rate limit and/or ban certain nodes that are doing things that are clearly spam or other malicious behavior.

I think no matter how efficient the nodes are and no matter how high the PoW is there’s always the possibility someone can abuse the network and those items make it less of an impact and more expensive but not impossible.

I think things to consider are:

  1. Limiting precomputing
  2. Rate limiting blocks (similar to how vote requests are currently limited). There needs to be a way to differentiate new blocks from republishing (eg using the PR nodes as gatekeepers where all blocks go through PR’s first and PR’s rate limit publish blocks to them)
  3. Banning nodes that abuse #2 or attempt to get around #1

Dynamic network wide PoW used to drop blocks is difficult. Not sure telemetry can be trusted and for dependent blocks in a chain or the source for a receive are required before those can be processed, so dropping blocks is extremely challenging to do in a way that doesn’t potentially break things more.

3 Likes