Getting pending-blocks

The pruned-node from NF is getting closer to the final release. However, that type of node still needs to download all the ledger, then delete the data after the complete verification.

There's some work in the "top-to-bottom" bootstrap, which I think is preferable. But, in order to take the full advantage of the "top-to-bottom", we need some way to get all pending-blocks.

Have two specific issues:

  1. Getting those blocks:
    Currently: the bulk_pull_account returns the hash (not the block itself), and we need the previous-block o to calculate the amount (ignoring legacy-blocks, which requires more work). Also, it only returns based on the receiver public-key, and we can't trust on the "balance" field.

  2. Checking if the given block is pending (or not):
    Currently: impossible (only "possible" if you are the receiver, which doesn't apply in that case).


A) The most "nano way" to solve that (which similar to already existent):

  • Create some new bulk_pull_pending. That message will be similar to bulk_pull. However, it may work in both directions and returns two blocks for each pending-block. You can get pending blocks using the receiver public-key (equal to the current bulk_pull_account), or the sender public-key.
  • Create new type on confirm_req (or new message) to verify if the blocks are pending or not.

Request the blocks using bulk_pull_pending then request votes for each one using such a new voting system

Pros: It's similar to how Nano works.
Cons: It will be a lot of votes, for each one for a single block.

B) The way that I initially thought:

  • Create new message like bulk_pull_signed_pending, which accepts the receiver/sender public-key and then return the "sender public-key", "receiver public-key", "hash" and "balance". However, it also returns the "representative public-key" and the "representative signature".
    That message must signed using the representative key. Because of the signature, we have an prove that those blocks are pending. If some account has 100 pending-blocks, we don't need to request 100 votes, since we have one vote (64 bytes) that proves all blocks. The vote may include some timestamp.

Request blocks using the bulk_pull_signed_pending and verifies the single signature. Of course, check if the node trusts on that representative.

Pros: It's a lot fewer votes compared to "A Method"; Doesn't need to parse legacy blocks (since it has the "balance" and it's signed/trusted).
Cons: It may have some performance impact (but old/inactive accounts could have the signature cached). It needs to request that same can against multiple nodes (in order to verify if many representatives agrees with the data).

C) The "snapshot way":

It needs some consensus to sync, which seems hard to do. It would be the optimal way to get a large number of pending-blocks for inactive accounts.

It's just ideas, and I tried to keep as simple as possible. I hope that it is understandable. I think it's possible to tweak some stuff, mainly for inactive accounts. For instance, for inactive accounts, it would be possible that the confirm_req accepts the hash of all blocks (like a resulting hash of the hash chain), instead of each one single block.

Also, it's possible to mix both methods to avoid download of duplicated data using bulk_pull_signed_pending. I mean, request the bulk_pull_signed_pending only one time. Then you can request others nodes, using some sort of confirm_pending_req, but using one single hash for all content (instead of 100 hashes, for instance).

I don't know if NF already have something similar going on. Or, already have better solutions.