IoTeX Network Rosetta Gateway Implementation | Data API

Overview

This repository (https://github.com/iotexproject/iotex-core-rosetta-gateway) implements the Rosetta for the IoTeX blockchain.

Run on Server

To build and run on a server:

make

To run tests:

make test

To clean-up:

make clean

Build and Run with Docker

To build the Docker image:

docker build -f ./docker/Dockerfile . -t iotexproject/iotex-core-rosetta-gateway

To run the Docker image:

docker run -p 8080:8080 -e "ConfigPath=/etc/iotex/config.yaml" iotexproject/iotex-core-rosetta-gateway

Examples for Accessing Testnet via Rosetta API

Account Balance

curl -X POST --data '{
    "network_identifier": {
        "blockchain": "IoTeX",
        "network": "testnet"},
    "account_identifier": {
        "address": "io1vdtfpzkwpyngzvx7u2mauepnzja7kd5rryp0sg"
    }}' https://rosetta.testnet.iotex.one/account/balance
#response:
#{"block_identifier":{"index":4051388,"hash":"b2a68ad3dcc3c61ad6a0cec3431400a40a195def87251c9c60920d7fcf973f2f"},"balances":[{"value":"11989999999999999999","currency":{"symbol":"IOTX","decimals":18}}],"metadata":{"nonce":1}}

Retrieve A Block

curl -X POST --data '{
    "network_identifier": {
        "blockchain": "IoTeX",
        "network": "testnet"},
    "block_identifier": {"index": 390873}}' https://rosetta.testnet.iotex.one/block
#response:
#{"block":{"block_identifier":{"index":390873,"hash":"5c084459315fcf0839ed9f2d8b89ca8fb039695a56007a071e5ce9d3c8908d95"},"parent_block_identifier":{"index":390872,"hash":"3ae76de97535f4908d7dd6b2d5f232543b1e5a9fe80a0e9d8f91fdd27d9363eb"},"timestamp":1573620900000,"transactions":[{"transaction_identifier":{"hash":"b37d5db44bd3dc182617b56744e12cab94486808eae1dc401599b611ed388164"},"operations":[{"operation_identifier":{"index":0},"type":"fee","status":"success","account":{"address":"io1ph0u2psnd7muq5xv9623rmxdsxc4uapxhzpg02"},"amount":{"value":"-10000000000000000","currency":{"symbol":"IOTX","decimals":18}}},{"operation_identifier":{"index":1},"type":"transfer","status":"success","account":{"address":"io1ph0u2psnd7muq5xv9623rmxdsxc4uapxhzpg02"},"amount":{"value":"-10000000000000000000","currency":{"symbol":"IOTX","decimals":18}}},{"operation_identifier":{"index":2},"type":"transfer","status":"success","account":{"address":"io1vdtfpzkwpyngzvx7u2mauepnzja7kd5rryp0sg"},"amount":{"value":"10000000000000000000","currency":{"symbol":"IOTX","decimals":18}}}]}]}}

Send A Signed Transfer

curl -X POST --data '{
    "network_identifier": {
        "blockchain": "IoTeX",
        "network": "testnet"},
    "signed_transaction": "0a470801100118c0843d220d31303030303030303030303030522e0a01311229696f316c397661716d616e776a3437746c7270763665746633707771307330736e73713476786b6532124104ea8046cf8dc5bc9cda5f2e83e5d2d61932ad7e0e402b4f4cb65b58e9618891f54cba5cfcda873351ad9da1f5a819f54bba9e8343f2edd1ad34dcf7f35de552f31a41d53b8aa4b0165326dcf2eddf4da1fcba8e864f805426c73ee7e73748713c48774bf117f7e78f18459645386ecbb644ca3cca89069920b20ff405768d3d1d6bb301"
}' https://rosetta.testnet.iotex.one/construction/submit
#response:
#{"transaction_identifier":{"hash":"292cda920534be56c78d6f13686dc7dbb94b77714b93abefb9f1e18679e2ae27"}}

Stake IOTX Into A Bucket

curl -X POST --data '{
    "network_identifier": {
        "blockchain": "IoTeX",
        "network": "testnet"},
    "block_identifier": {"index": 4034780}}' https://rosetta.testnet.iotex.one/block
#response:
#{"block":{"block_identifier":{"index":4034780,"hash":"bc1ad74d423f84e553602798e86019254b70d4499f1738a11c285ab9e31ea3b2"},"parent_block_identifier":{"index":4034779,"hash":"ac25b97cb7c9743b496cf45586d442ae4777753c77ca08d539bb96f30bca08c6"},"timestamp":1592801095000,"transactions":[{"transaction_identifier":{"hash":"9f261c47ad6611388c8e4569d2db378d2a7d98607c4259f5f9819ae6703742e6"},"operations":[{"operation_identifier":{"index":0},"type":"fee","status":"success","account":{"address":"io1mflp9m6hcgm2qcghchsdqj3z3eccrnekx9p0ms"},"amount":{"value":"-10000000000000000","currency":{"symbol":"IOTX","decimals":18}}},{"operation_identifier":{"index":1},"type":"stakeCreate","status":"success","account":{"address":"io1mflp9m6hcgm2qcghchsdqj3z3eccrnekx9p0ms"},"amount":{"value":"-100000000000000000000","currency":{"symbol":"IOTX","decimals":18}}}]}]}}

Register A Delegate

curl -X POST --data '{
    "network_identifier": {
        "blockchain": "IoTeX",
        "network": "mainnet"},
    "block_identifier": {"index": 5160923}}' https://rosetta.testnet.iotex.one/block
#respone
#{"block":{"block_identifier":{"index":5160923,"hash":"fe653d92713bbd0f438fb8426edc3daab10a2eef28aa95d81fdaab460dd52f8e"},"parent_block_identifier":{"index":5160922,"hash":"40a7de13ef6f183c6c5dbb4bcb18aad3cc78e56c0c2ced1c1e1602bc4cc5c5e9"},"timestamp":1591033570000,"transactions":[{"transaction_identifier":{"hash":"a7e60af13e75646293c412b96e959d5edb8c254d3a1bcd9b6d7536a1af37e7c6"},"operations":[{"operation_identifier":{"index":0},"type":"fee","status":"success","account":{"address":"io180w8cmhrxpjjulry2g4zpyhm8mx933yhqkr4g6"},"amount":{"value":"-10000000000000000","currency":{"symbol":"IOTX","decimals":18}}},{"operation_identifier":{"index":1},"type":"candidateRegister","status":"success","account":{"address":"io180w8cmhrxpjjulry2g4zpyhm8mx933yhqkr4g6"},"amount":{"value":"-2000000000000000000000000","currency":{"symbol":"IOTX","decimals":18}}}]}]}}

More examples are here - https://github.com/iotexproject/iotex-core-rosetta-gateway/blob/master/README.md

About IoTeX

Founded as an open-source platform in 2017, IoTeX is building the Internet of Trusted Things , an open ecosystem where all “things” — humans, machines, businesses, and DApps — can interact with trust and privacy. Backed by a global team of 30+ top research scientists and engineers, IoTeX combines blockchain, secure hardware, and confidential computing to enable next-gen IoT devices, networks, and economies. IoTeX will empower the future decentralized economy by “connecting the physical world, block by block”.

Hi IoTex Team!

Thanks for sending over your Rosetta Implementation for review! We have identified a few issues listed below and will review again once these are fixed. Please let us know if you have any questions around any feedback!

Deployment Issues

1. Dockerfile not buildable anywhere

In the Dockerfile, we saw you COPY multiple times from the build context (the repo where you are building the container). As noted in the docs, your Dockerfile must be buildable anywhere and this is not possible if you COPY files into the container. You should clone your repository at a particular hash in the Dockerfile.

It must be possible to build your Dockerfile from any location (i.e. run docker build). For this reason, your Dockerfile must not copy any files from the directory where it is built. Instead, any required files must be downloaded from the internet with explicit versioning.

Please see docs here

2. Storage Location Not Specified

The instructions to run your node don’t mention which directory to mount to an external volume (ex: /data). This issue may be related to issue 3 (see below).

All persistent data must be written to the /data directory. If the node you are working on utilizes a database (like Postgres) to store data reliably, ensure this is configured to save information to this directory (and in a manner that would not corrupt any data stored by the core node process or other necessary services). It must be possible to stop all services defined in the Dockerfile and restart them (without corruption) using only the state stored in this /data directory.

Please see docs here

3. Blockchain Node Not Running In The Same Dockerfile

When running your implementation, we didn’t see any syncing logs get printed after starting up the node. We took a look at your code and suspect that your implementation is connecting to an external service to query the node data. We noted this is not allowed in the docs.

Any implementation of a Rosetta Server must connect exclusively to a running blockchain node in the same Dockerfile. It is not ok to connect to some external service that preprocesses blockchain data.

Please see docs here

4. /network/status Returns No Peers

Requests

curl -X POST --data '{
    "network_identifier": {
        "blockchain": "IoTeX",
        "network": "testnet"},"metadata": {}}' http://localhost:8080/network/status

Response

{"current_block_identifier":{"index":4164697,"hash":"810d17d64d616fb7b9399ae0dfae5a6c987c66a72166edb9ffa33bebc1562d83"},"current_block_timestamp":1593469675000,"genesis_block_identifier":{"index":1,"hash":"663fc0a40a4943f1b56f501aee3ad626b5396e850aa53c5bd8759d0d47694dfc"},"peers":null}

rosetta-cli Check Issues

1. Incorrect Test

After reading your test.sh file, we saw you only tested the first 10 blocks. We believe you based your test off of this test but you didn’t spin up a fake testnet and try all operations in the first 42 blocks. If you are just running an integration test against your existing testnet, you should sync all the way to tip

You probably don’t want to specify --end 10 in your test. You must also set --lookup-balance-by-block=false not --lookup-balance-by-block false (fortunately this is going away with the v1.4.0 spec).

./rosetta-cli check --lookup-balance-by-block=false --bootstrap-balances ./bootstrap_balances.json --block-concurrency 8

2. High Error Rate on Fetch (Exhausted retries)

We saw a lot of errors when running the rosetta-cli check and could not get past the first few thousand blocks without exceeding our error retry limit. We believe this may be related to using a public node for your implementation.

 rosetta-cli check --lookup-balance-by-block=false --bootstrap-balances tests/bootstrap_balances.json --block-concurrency 8

Error

Check failed: retries exhausted: unable to fetch block {
 "index": 453
}: unable to sync to 1000

Thank you very much for your thoughtful review, Qi.
Here are some thoughts from our side regarding to the issues you listed above.

For Dockerfile, from developer point of view, it is nature to build docker image which represent local state of the repo. IMO, clone git repo in dockerfile is pretty similar to clone a tagged branch and build docker file from there or use a tagged docker image directly.

I respect that put dockerfile this way will highly simplify your deployment process. Therefore, we will put another Dockerfile in this matter and keep our existing dockerfile for our development purpose. We will also make sure we put our node process in that Dockerfile to resolve issue 2 and 3 you listed.

For issue 4, unfortunately our chain native API doesn’t expose node peer information, so at current stage, I think we have to leave them empty.

About testing, we will restructure the way we test the gateway without connect it to our mainnet/testnet api gateway.

Will get back to your guys, once we address the issues.
Thank you!

IoTeX core-dev has made a lot of progress on the integration. Concretely:

  1. Docker issues for deployment has been addressed
    – run without depending on current repo status
    – the storage location is specified
    – chain node is running with a gateway in the deployment environment

  2. The test has been changed to be a local e2e test with mock transactions

  3. IoTeX mainnet and testnet have the new binary (v1.1.0) deployed and have been synced from scratch. All checks with rosetta-cli are successfully passed indicating correctness and reliability of the transactions.

In addition, we are actively working on the Construction API and it will be ready very soon.

@qi_wu @patrick.ogrady Please take a look and suggestions/comments are welcomed!

IoTeX Mainnet node with Rosetta Gateway docker deployment guide can be found here: https://github.com/iotexproject/iotex-core-rosetta-gateway/tree/master/docker/deploy

Thanks for the update! We will take a look and let you know!

1 Like