While a rank of representatives is possibly not desired, at least recognizing outliers and bad performs across the top representatives is useful.
So far, the community has explored the idea of average vote latency as a quantitative score for representatives, but also categorizing into a qualitative measure of "Fast", "Slow", etc.
This measure does unfortunately come with several drawbacks:
It is affected by network latency, making it unfair. This can be solved if using many antennas across multiple locations
It is supposed to be a way to measure how representatives perform under load, however, it would require the measuring nodes to be able to keep up with spam perfectly
Despite the drawbacks it has been helpful in some cases through the implementation mynano.ninja, e.g., https://mynano.ninja/account/nanocenter.
The Nano Foundation doesn't consider it our place to be defining possible measures to score representatives, however our knowledge and experience with the protocol can help in finding a better approach.
We know of some current experimentation with scoring representatives by selectively requesting votes for blocks and seeing if the representative can keep up with the request rate. This can then be used to analyze how many confirmation requests were dropped because the representative could not keep up.
This can be done, for example, during a spam event, by measuring across all reps and all blocks, on average we should see that the slowest representatives perform consistently worse on this measure. Alternatively a change in the node could be made to repeatedly request confirmation even for confirmed blocks. Without any kind of rate-limiting per-channel this should be feasible.
Initial results using this measure on beta network during a spam event show that the slowest representatives do indeed have a larger percentage of dropped packets (per-channel tcp_write_drops).
Are there any thoughts on this approach? One related question is whether this would be compatible with the DHT in the future.