Status
IN PROGRESS
Stakeholders
Makoto Takemiya Egor Ivkov Iurii Vinogradov Vladislav Markushin 
Outcome

Unable to render Jira issues macro, execution error.

Due date
Owner
Nikita Puzankov 

Background

Previous assumptions about Iroha Network and Web API in general were usage of iroha-client with encapsulation of underlying (plain TCP) protocols.
New needs from dependent projects require discussion of these assumptions and decision how to work with iroha network.

Iroha Bridge module provides an ability to connect different blockchains together and needs an ability to communicate between both sides via web interface.

Current Iroha approach is a proprietary TCP based protocol encapsulated inside `iroha_network` crate. `iroha_client` crate uses `iroha_network` inside and should be  used itself by other applications including other blockchain solutions like `Polkadot`.

It turns out that `Substrate's` off chain workers provides only HTTP (more high level) API for communications with external resources (like Iroha Peer).

Requirements from Iurii Vinogradov

  • HTTP compatible Iroha Peer API
  • HTTP client library for Instructions, Queries, Events
  • Rust client library (for Substrate's Off-chain workers)
  • JS Client library

Problem

Different clients environments have different restrictions - the strictest one is Substrate's Off-chain worker runtime which only supports HTTP 1.

Also DevOps team and Client side developers prefer to use JSON as a client-peer communication format. Some scenarios require to have an ability to combine Instructions API with Events API (send transaction and start consuming updates on it).

Solution

As a solution we propose this high-level design. HTTP and WebSocket protocols will be used for communication between Clients and Iroha Peers.

Iroha should provide HTTP API.

  • Endpoints:
    • Iroha Special Instructions
    • Iroha Queries 
    • Iroha Events (under consideration - needed for Substrate)
  • MIME type of the HTTP API will be `application/json`
  • Clients build their own functionality or use 3rd party clients to establish HTTP connection, sign and construct transactions
  • JSON RPC or RESTfull approach should be discussed

Additionally Iroha should provide WebSocket API.

  • Endpoints:
    • Iroha Events
    • Iroha Special Instructions (under consideration - was requested by client-side developers)
    • Iroha Queries (under consideration - was requested by client-side developers)
  • JSON RPC can be used as high level protocol over WebSocket

Decisions

  • Iroha will provide HTTP and WebSocket API for clients
  • JSON format for messages encoding/decoding will be used

Alternatives

  • Stay with TCP and use middle ware for protocol and messages transformations - not user friendly
  • Use HTTP 2.0 - will not work with Substrate

Concerns

  • WebSocket connections should have configuration options to prevent bad performance of Iroha peers under high load

Assumptions

  • Bridge clients will use Substrate's Off-chain Workers with an ability to communicate over HTTP only

Risks

  • Performance regress `[8; 5]`

Action items

31 Comments

  1. Iurii Vinogradov

    > Iroha Bridge has an additional application as a proxy on top of Iroha API (as a standalone app or service mesh)

    I think the standalone app is not an Iroha approach. Might be implemented as a module on  iorha2. We also can expect that some blockchains which use other protocols like Protobuf might be interested to connect iroha2, so the module approach seems someway better because it allows to add/exclude networking protocols.

    > Iroha Bridge module provides HTTP wrapper on top of Iroha API (on the Peer side)
    If the bridge will provide only bridging functions HTTP interface then another HTTP server might be needed if we will need to provide HTTP API for other functions. So the Iroha HTTP server is better to be decoupled from concrete functionality to have the ability to support all need functions for other possible cases.

    So my vote is for the dedicated HTTP server module on Iorha2 not included in the core but has an ability to proxy all ISI and queries if needed.

    1. Nikita Puzankov

      So my vote is for the dedicated HTTP server module on Iorha2 not included in the core but has an ability to proxy all ISI and queries if needed.

      Looks good, let's wait for Makoto Takemiyareview and we will need to plan this feature with Vadim Reutskiy.

  2. Iurii Vinogradov

    >Decisions


    > Iroha Bridge module will provide additional HTTP Endpoints on top of existing Torii Endpoints
    Nikita Puzankov Why you finally decided on this option if I described disadvantages in the message above and proposed another solution which is also included in the list. You did not mention any reasons why you made this decision and you did not discuss it with us. 

    1. Nikita Puzankov

      Iurii Vinogradov

      1. It's RFC, Request for Comments - I just proposed this option as a primary because we agreed on it. And it is the solution that we discussed here Re: Iroha Network API, because it is mandatory for Bridge and DEX modules it should be packed with them. Maybe you was confused by part "Iroha Bridge module"?
      2. All the reasons described in the RFC and RFC is a discussion - so please, discuss and comment (smile)

      1. Iurii Vinogradov

        As I remember we agreed on that it will be in a separate module. And it was discussed in first comment to this RFC.
        The HTTP server is not a part of the bridge. It is a separate functionality. It also will contain not only Bridge ISI, but generally all user ISI. It will be used by many modules - DEX, Bridge - it is already two different parts. I think that our future web wallet or even other types of wallets and apps also most probably will have the intention to use it. So, the inclusion of HTTP code to the bridge module seems not to make sense. 

        1. Nikita Puzankov

          Rewrote

  3. Iurii Vinogradov

    >Decisions


    > Cloud Events will be implemented via HTTP instead of WebSocket because of Substrate's restrictions
    Nikita Puzankov It is also not correct. substrate supports WebSockets as Salakhiev Kamil mentioned. So we don't have any restrictions here from Substrate side. 

    1. Nikita Puzankov

      You may ask Salakhiev Kamilfor proofs, but according to Off-chain workers documentation (I will paste a link one more time here https://substrate.dev/docs/en/knowledgebase/runtime/off-chain-workers#fetching-external-data) there is no WebSocket support. As I asked Alexander Lednevin the first issue about client for `no-std`  Unable to render Jira issues macro, execution error.  more than a month ago about restrictions and your choice for runtime, as I ask you now - please, give me restrictions and supported abilities of your clients environments, because we already spend a lot of time discussing things without them (smile)

      1. Iurii Vinogradov

        Ok, I also found only proof that Polkadot has websocket as a connection to the node, but did not found that we can use it from the pallet or from the web worker. Salakhiev Kamil Can you help to find some docs on how to use web sockets from the pallet or web worker?
        https://www.substrate.io/kb/runtime/metadata - use Web socket to connect to the node.

        We will need subscriptions for the Web also. I think while we decided to implement JS library independently it will be also easier for them to use WebSockets for subscriptions. For example, WEB should have the ability to be subscribed to transaction status changes.

        1. Iurii Vinogradov

          Comment from Kamil'

          Indeed we cannot use websockets from pallets, only http. When I said that ws and http are supported I meant that Polkadot’s RPC supports http and ws transports.

        2. Salakhiev Kamil

          For subscription Web Sockets transport should be implemented. As far as I know HTTP would not allow to implement subscription mechanism

          1. Nikita Puzankov

            But it cannot be used by off-chain workers, why do we need web sockets then?

            1. Salakhiev Kamil

              HTTP transport is not enough to implement subscription

              1. Nikita Puzankov

                But if it's the only API available, what would we do?)



                1. Salakhiev Kamil

                  Support two transports with the same API.
                  First priority would be HTTP, later we can introduce WebSockets subscription mechanism

                  1. Salakhiev Kamil

                    The way it is done in Kagome and Polkadot is that we have two RPC ports: for http and websocket requests. Both use JSON-RPC api for processing requests. If you are interested such apis are described here: https://github.com/w3f/PSPs/blob/psp-rpc-api/psp-002.md.
                    I would love to have similar document for Iroha2

                  2. Nikita Puzankov

                    Salakhiev Kamil once again - who will need WebSockets? We are talking about HTTP API only for Substrate's off-chain workers, because they support only HTTP - why we need to add WebSockets if they are not supported?

                    1. Salakhiev Kamil

                      use cases:

                      1. Subscribe to transaction status (stateless validated, stateful validated, committed or similar statuses)
                      2. Subscribe to new blocks
                      3. Subscribe to transfer transactions

                      1. Nikita Puzankov

                        No, it's functional requirements for "streams api" - who will use them and why it should be implemented via WebSockets?

                        1. Salakhiev Kamil

                          Those streams apis will be used by clients. For example block explorer needs to be subscribed to the new blocks.
                          Read this article to understand why WebSockets. In short it allows avoid continuous polling + performance is higher than other solutions. If you have other options what transport to use we can discuss

                          1. Nikita Puzankov

                            We are trying to keep 2.0.0 requirements at minimum. So if we do not have such a clients, let's postpone it to 2.1.0.

                            1. Salakhiev Kamil

                              As long as projects using Iroha2 are okay with it I don't mind to postpone it. Iurii Vinogradov when do you think it will be critical to have streaming over websockets?

                              1. Nikita Puzankov

                                I think Iurii Vinogradovand Egor Ivkovwill check abilities of WASM and WASI for browsers and will provide requirements on monday

                                1. Nikita Puzankov

                                  As all this options were described here:  Unable to render Jira issues macro, execution error.


          2. Iurii Vinogradov

            For the prototype we can use HTTP polling, but it will not be very efficient for Production, so we will need to think ways to handle it. 

            1. Nikita Puzankov

              It will be not enough because workers need to receive this information in real-time?

  4. Salakhiev Kamil

    For REST vs JSON-RPC question here is what is used in some other blockchains:


    ## Substrate based chains

    JSON-RPC on top of Websockets and HTTP
    1. API: https://github.com/w3f/PSPs/blob/psp-rpc-api/psp-002.md
    2. Subscription events by the means of Websockets

    ## NEAR

    JSON-RPC on top of HTTP
    1. API: https://docs.near.org/docs/interaction/rpc
    2. No subscription events, continous polling for statuses

    ## Tezos

    REST API
    1. API: https://medium.com/chain-accelerator/how-to-use-tezos-rpcs-16c362f45d64
    2. No subscription events

  5. Egor Ivkov

    Currently the minimal http server is implemented, but there are several questions of features that we might or might not have in our MVP:

    1. Should we provide Transfer-Encoding header? In particular chunked transfer coding helps to transfer big amounts of data, by dividing the message into chunks. In general it is required to be compliant with the HTTP/1.1 standard. But depending on the use cases that we plan, it might be a good decision to leave it out of the scope of the MVP.
    2. Do we need to support HTTP over TLS (HTTPS) in MVP? As an alternative a reverse proxy (e.g. nginx reverse proxy) can be set up over our server. Also it is not clear if we will need to encrypt out http messages at all at this level.

    1. Nikita Puzankov

      Hi Egor Ivkov,

      1. I think we can do not provide chunks for now and implement it if needed later.
      2. I also think that reverse proxy would be enough, but maybe DevOps team can make it more clear - Vadim Reutskiy, please ask them to comment here.


      1. Vadim Reutskiy

        1. I have the same thoughts, let's prepare to implement them later.
        2. There is nothing to discuss with our DevOps, actually, it depends on the requirements, too. I will add that question to our questionnaire for stakeholders.

      2. Vadim Reutskiy

        However, seems like that we can also postpone the HTTPS/TLS integration and back it up now by the reverse-proxy as mentioned.