P2P network

Background

P2P-network layer is slow, and lacks important parts for general purpose blockchain platform.

Problem

The main problem is the very slow creation and closing of a connection for every request, in particular if it is a distributed network around the world, and round-trips can be like 300-350ms for every packet. As you know, every connection needs to toss 3 packets between client and server on opening. So, every request may take about half a second. This is unacceptable for serious financial distributed system.

Solution

I propose several points of improvement:

1. Using bootstrap nodes to propagate initial network information - peers and their public keys if needed (already has a task: Unable to render Jira issues macro, execution error. ).
2. Adding genesis block hash for peers to know with which peers to work. This is similar to having a password to connect to network. If peer knows bootstrap IP:port and genesis hash it can connect and sync blocks, if not - it will not make successful connection to this bootstrap.
3. When we have bootstrap node/nodes, that propagate initial network info, then we can easily make it universal and implement it in a form of Peer Exchange.
4. To resolve the main problem we need to implement holding some number of peer connections, capped to some constant or config value. Around 15-20 connections will be enough for gossip protocol, and nodes will not search/request additional peers from bootstrap and any other nodes.
5. It would be better to implement gossip protocol for transactions & block propagation, but current consensus protocol needs to send them to particular peer, don't know how can we circumvent this.
6. Message format and encoding. If we want this system to be efficient, we can encode every message as Enum variant, and serialize/deserialize to/from CBOR format. It is very compact and useful.
7. Traffic encryption. It is imperative to encrypt blockchain communication between the nodes to protect financial data from eavesdropping and MitM-attacks. So, I propose to implement Diffie-Hellman hand-shakes based on Elliptic Curves (x25519) and encrypting messages by ChaCha20Poly1305. For further DPI mitigation we can add some random garbage to handshakes as is done in Noise protocol. This approach will eliminate need of VPN.

From the architectural view every node will hold one server (listener) socket for other nodes to connect, and several outgoing connects to other nodes.

Decisions

1. Every node will hold 2 sets of peer-connections: validators and unprivileged peers.
2. Every validating node is trying to connect to every other validating node (for current round/epoch).
3. Using SCALE codec for serialization, not CBOR.
4. Using proposed encryption: Diffie-Hellman hand-shakes based on Elliptic Curves (x25519) and encrypting messages by ChaCha20Poly1305, the Ursa has all primitives for it. No one was against adding some garbage to handshakes.
5. Trusted peers set is not the same as validators. Trusted peers can be different for each node and it means that this set can be used as a bootstrap. But validators should be encoded in the genesis block and later added with AddPeer.

Alternatives

I think the alternative would be using libp2p, but it doesn't give such flexibility, encryption and robustness as using bare sockets and mio, for example. Do we need the DHT anyway?

Concerns

1. The implementation of addressable peer connections in such system for sending messages to concrete peer is a bit concerning. In particular if we talk about public network and that peer is behind NAT.
2. Is it possible to use an Enum for every type of message? We need to discuss this.

Assumptions

I assume that we will be able to implement gossip protocol for current consensus type.

Risks

Risks of flooding the node. Anyone can flood the node with fake handshakes to incline the node to compute Diffie-Hellman common key and then close the connection and retry. But this type of vulnerability is on every server with an HTTPS-server, or every node of secure messenger.

Additional Information

This type of network implementation is already implemented by me in my pet-project: https://github.com/Revertron/Alfis