[OUTDATED] Fetching Blocks using Celo's Data API Implementation

What better way to learn about Rosetta than fetching a few blocks from a Rosetta Data API implementation?

In this blog post, we will start Celo’s Data API implementation and view
a few blocks on mainnet using both the rosetta-cli and rosetta-sdk-go.
It is our hope that, by the end of this blog post, you will feel comfortable fetching blocks from any Rosetta Data API implementation!

Prerequisites

To follow along on this blog post, you’ll need basic proficiency in the terminal (the demo takes place entirely on
the command line) and some basic software installed on your computer:

  1. Install Docker Desktop
  2. Install Golang >= 1.13

Start Celo Node

Create Start Script

First, we will create a simple script file called run_node.sh which will be used to start the Celo Data API implementation with
the default settings. Put the following code into this file:

#!/bin/bash

# Create a directory to store Celo blockchain data
export DATADIR="celo-data";
mkdir -p "${DATADIR}";

# Get mainnet genesis file
curl 'https://storage.googleapis.com/genesis_blocks/rc1' > "${DATADIR}/genesis.json";

# Pull pre-built rosetta image (although the build URL contains 'celo-testnet' it is also used for Celo mainnet)
export CONTAINER_NAME=us.gcr.io/celo-testnet/rosetta:0.6.2;
docker pull ${CONTAINER_NAME};

# Start Celo node + Rosetta Data API implementation in single container
export CELO_BOOTNODES="enode://5e0f4e3aaa096e2a2db76622b335cab4d3224d08d16cb11e8855a3a5f30c19d35d81a74b21271562e459495ab203c2f3a5a5747a83eb53ba046aeeb09aa240ff@34.83.110.24:30303,enode://5c9a3afb564b48cc2fa2e06b76d0c5d8f6910e1930ea7d0930213a0cbc20450434cd442f6483688eff436ad14dc29cb90c9592cc5c1d27ca62f28d4d8475d932@34.82.79.155:30301,enode://d810b00775d3cc8b20fc0bb6fa1efefc222f56263bcc21109df714d856df5165eb41a5ad48603dc8108f5c09f103899a56fdae1d6756cc130d5222ab102ebd90@35.233.182.101:30303,enode://7dadc03eb447b8c73012bb194324e96ed75c26f72e0d8a2e5f5016f630601843cd14c4801b8d9b9d34a8d53ac0165f45033212d1027f3fd814ba39693f2af526@176.125.234.112:30303,enode://27dec4743edb339622d8524d0b65ec8725f7c16cfc8335ad1966d522c47e3a2563db47be71efc861c6407471c3774583f18a352d059b686d81f426510cc661e4@176.125.234.111:30303,enode://caa2be7a0c28f91778074f089274b5e027a495f5f61ecc5d1090495142b22b3ba74af667e7f1b899720c51ad98d26210a400cc995da4bf2d9ebbd47846904e34@95.216.127.4:30303,enode://13a830a823efb51853ff1bc60ff11fe683b3519f5460dc78b1c4dbb319503e3bb3fb70242d988f27c9fb9b001ff8684f85cedb64a03c7de8659b7ce4e91d6ea1@91.90.43.7:30303,enode://97cc32024068eaed4907bb46895ffcb6bd29794939147f3a252dba4e402b5d43ea254ee0cd0c8eb12f50fdb5cfea0fc220378fd1136b8a302ea327223f3e6350@35.247.57.23:30303,enode://5180aa5e77289b4429888e79c7d0c9e8cda8bf9c2604ed114d4829d443bc61db71fa8e8c46c4eaa7eac17030f51dd5834b75d6634deb197d515d4f19d31787d6@34.83.8.6:30303,enode://4c393c7ee1555b12c11377d3722a7b37e68fcdadc11e4da28c8ad4b28367422a2890c48874c167e128f3bb1edff319a4cdbc327b0218246be51750014bee1297@34.83.255.200:30303,enode://ccd8d98c065923922bd929d65f6e2ec005acac45876b7b4dd3b6c89dac5b5ae8664003238297a70bd5e75290dba797ed0558057226d6af868026263620e32868@35.233.237.49:30303,enode://f65013f1ac6827e275c2d2737ce13357f620d4364124d02227a19321c57f8fbf9214a9411de49d49f180b085b031d9d23211a6ead4499fc5f9d3592b55322123@50.17.60.161:30303,enode://f9484ead9f7ac4797e1db19a8258c5356be8dc617e1f91c4748f7fa9ba6ede65f70261a7a1052cef7ae84d9a615a5b97a07ef0c6f3c20e4292279fc94412a7a8@35.230.3.194:30303"

docker run --rm \
  -v "${PWD}/${DATADIR}:/data" \
  -p 8080:8080 \
  "${CONTAINER_NAME}" \
  run --geth.bootnodes "${CELO_BOOTNODES}"

Make sure to run chmod 755 run_node.sh after creating the file so your computer can run the script.

Start the Node

Next, let’s start the node! You should run the following in your terminal: ./run_node.sh. Note, it may take
a few minutes for your node to start syncing.

Before moving on to the next section, make sure your node syncs to at least block height 3550 (should
take a few minutes at most). We fetch this block in later steps and you will
receive an error if your node has not yet synced it. You should see a series of logs printed out that
indicate the most recent block synced (these logs indicate that up to block 4000 has been synced):

INFO Core Contract Address Changed            srv=celo-monitor pipe=processor       name=DoubleSigningSlasher newAddress=0x50C100baCDe7E2b546371EB0Be1eACcf0A6772ec txIndex=0
INFO Core Contract Address Changed            srv=celo-monitor pipe=processor       name=DowntimeSlasher      newAddress=0xeFE50f83bA23240A85c39afF429b31e556d2C80F txIndex=0
INFO Gas Price Minimum Updated                srv=celo-monitor pipe=processor       from=100000000 to=100660086 block=3093
INFO Gas Price Minimum Updated                srv=celo-monitor pipe=processor       from=100660086 to=100000000 block=3094
INFO Core Contract Address Changed            srv=celo-monitor pipe=processor       name=Governance           newAddress=0xD533Ca259b330c7A88f74E000a3FaEa2d63B7972 txIndex=0
INFO Stored 1000 blocks                       srv=celo-monitor pipe=persister       block=4000 registryUpdates=0

Fetch Blocks

Now that we have a Celo Data API implementation running, we can fetch a few blocks and their transactions.

Make sure to open a new terminal window before walking through the following examples or you will stop the Celo node you started.

rosetta-cli

First, we will fetch a block using the rosetta-cli. The rosetta-cli is
a tool developed by Coinbase to debug Rosetta implementations and make performing common queries easy (like fetching
a block at a given height).

Install rosetta-cli

To use the rosetta-cli, you must first install it using go get:

go get github.com/coinbase/rosetta-cli

View Block 3530

After installing the rosetta-cli, run the following command to get the block at height 3530:

rosetta-cli view:block 3530

You should see a single transaction printed out with both fee and transfer operations:

Primary Network: {
 "blockchain": "celo",
 "network": "42220"
}
Current Block: {
 "block_identifier": {
  "index": 3530,
  "hash": "0xda6ff2e96553917e5694109b4e6fac686d20164b9eb2a222d7180e18d8b66ca3"
 },
 "parent_block_identifier": {
  "index": 3529,
  "hash": "0x1c4c285a72c010b3202f18c0fee2a2f396cb3567ed9de5946068b4e3c75cc8f8"
 },
 "timestamp": 1587589758000,
 "transactions": [
  {
   "transaction_identifier": {
    "hash": "0x6cc2eeee50a3fdd79f8b272d4b74d9270ed6e3372f67551c3afde3e8fc766485"
   },
   "operations": [
    {
     "operation_identifier": {
      "index": 0
     },
     "type": "fee",
     "status": "success",
     "account": {
      "address": "0xb952930a3656A9CbaB21Df5919F94C61A495bF79"
     },
     "amount": {
      "value": "8400000000000",
      "currency": {
       "symbol": "cGLD",
       "decimals": 18
      }
     }
    },
    {
     "operation_identifier": {
      "index": 1
     },
     "related_operations": [
      {
       "index": 0
      }
     ],
     "type": "fee",
     "status": "success",
     "account": {
      "address": "0xD533Ca259b330c7A88f74E000a3FaEa2d63B7972"
     },
     "amount": {
      "value": "2100000000000",
      "currency": {
       "symbol": "cGLD",
       "decimals": 18
      }
     }
    },
    {
     "operation_identifier": {
      "index": 2
     },
     "related_operations": [
      {
       "index": 0
      },
      {
       "index": 1
      }
     ],
     "type": "fee",
     "status": "success",
     "account": {
      "address": "0xE23a4c6615669526Ab58E9c37088bee4eD2b2dEE"
     },
     "amount": {
      "value": "-10500000000000",
      "currency": {
       "symbol": "cGLD",
       "decimals": 18
      }
     }
    },
    {
     "operation_identifier": {
      "index": 3
     },
     "type": "transfer",
     "status": "success",
     "account": {
      "address": "0xE23a4c6615669526Ab58E9c37088bee4eD2b2dEE"
     },
     "amount": {
      "value": "-300000000000000000000",
      "currency": {
       "symbol": "cGLD",
       "decimals": 18
      }
     }
    },
    {
     "operation_identifier": {
      "index": 4
     },
     "related_operations": [
      {
       "index": 3
      }
     ],
     "type": "transfer",
     "status": "success",
     "account": {
      "address": "0x92AD020Cde6A4e566770C603ae8315a9d7252740"
     },
     "amount": {
      "value": "300000000000000000000",
      "currency": {
       "symbol": "cGLD",
       "decimals": 18
      }
     }
    }
   ]
  }
 ]
}
Balance Changes: [
 {
  "account_identifier": {
   "address": "0xb952930a3656A9CbaB21Df5919F94C61A495bF79"
  },
  "currency": {
   "symbol": "cGLD",
   "decimals": 18
  },
  "block_identifier": {
   "index": 3530,
   "hash": "0xda6ff2e96553917e5694109b4e6fac686d20164b9eb2a222d7180e18d8b66ca3"
  },
  "difference": "8400000000000"
 },
 {
  "account_identifier": {
   "address": "0xD533Ca259b330c7A88f74E000a3FaEa2d63B7972"
  },
  "currency": {
   "symbol": "cGLD",
   "decimals": 18
  },
  "block_identifier": {
   "index": 3530,
   "hash": "0xda6ff2e96553917e5694109b4e6fac686d20164b9eb2a222d7180e18d8b66ca3"
  },
  "difference": "2100000000000"
 },
 {
  "account_identifier": {
   "address": "0xE23a4c6615669526Ab58E9c37088bee4eD2b2dEE"
  },
  "currency": {
   "symbol": "cGLD",
   "decimals": 18
  },
  "block_identifier": {
   "index": 3530,
   "hash": "0xda6ff2e96553917e5694109b4e6fac686d20164b9eb2a222d7180e18d8b66ca3"
  },
  "difference": "-300000010500000000000"
 },
 {
  "account_identifier": {
   "address": "0x92AD020Cde6A4e566770C603ae8315a9d7252740"
  },
  "currency": {
   "symbol": "cGLD",
   "decimals": 18
  },
  "block_identifier": {
   "index": 3530,
   "hash": "0xda6ff2e96553917e5694109b4e6fac686d20164b9eb2a222d7180e18d8b66ca3"
  },
  "difference": "300000000000000000000"
 }
]
Transaction 0x6cc2eeee50a3fdd79f8b272d4b74d9270ed6e3372f67551c3afde3e8fc766485 Operation Groups: [
 {
  "Type": "fee",
  "Operations": [
   {
    "operation_identifier": {
     "index": 0
    },
    "type": "fee",
    "status": "success",
    "account": {
     "address": "0xb952930a3656A9CbaB21Df5919F94C61A495bF79"
    },
    "amount": {
     "value": "8400000000000",
     "currency": {
      "symbol": "cGLD",
      "decimals": 18
     }
    }
   },
   {
    "operation_identifier": {
     "index": 1
    },
    "related_operations": [
     {
      "index": 0
     }
    ],
    "type": "fee",
    "status": "success",
    "account": {
     "address": "0xD533Ca259b330c7A88f74E000a3FaEa2d63B7972"
    },
    "amount": {
     "value": "2100000000000",
     "currency": {
      "symbol": "cGLD",
      "decimals": 18
     }
    }
   },
   {
    "operation_identifier": {
     "index": 2
    },
    "related_operations": [
     {
      "index": 0
     },
     {
      "index": 1
     }
    ],
    "type": "fee",
    "status": "success",
    "account": {
     "address": "0xE23a4c6615669526Ab58E9c37088bee4eD2b2dEE"
    },
    "amount": {
     "value": "-10500000000000",
     "currency": {
      "symbol": "cGLD",
      "decimals": 18
     }
    }
   }
  ],
  "Currencies": [
   {
    "symbol": "cGLD",
    "decimals": 18
   }
  ],
  "NilAmountPresent": false
 },
 {
  "Type": "transfer",
  "Operations": [
   {
    "operation_identifier": {
     "index": 3
    },
    "type": "transfer",
    "status": "success",
    "account": {
     "address": "0xE23a4c6615669526Ab58E9c37088bee4eD2b2dEE"
    },
    "amount": {
     "value": "-300000000000000000000",
     "currency": {
      "symbol": "cGLD",
      "decimals": 18
     }
    }
   },
   {
    "operation_identifier": {
     "index": 4
    },
    "related_operations": [
     {
      "index": 3
     }
    ],
    "type": "transfer",
    "status": "success",
    "account": {
     "address": "0x92AD020Cde6A4e566770C603ae8315a9d7252740"
    },
    "amount": {
     "value": "300000000000000000000",
     "currency": {
      "symbol": "cGLD",
      "decimals": 18
     }
    }
   }
  ],
  "Currencies": [
   {
    "symbol": "cGLD",
    "decimals": 18
   }
  ],
  "NilAmountPresent": false
 }
]

For your convenience, the rosetta-cli also prints out grouped operations (i.e. operations that reference
each other with the related_index field) and all balance changes on the block (derived from the operations
present in the block). You can also view this block on Celo’s block explorer.

Unexpected Error?

If you see an an error message (like block 3530 fetch error: {Code:502 Message:Celo node rpc request failed Retriable:false}'),
check to see if your node has synced to at least block 3550. There is an example of what this looks like in the logs in this section.

rosetta-sdk-go

Next, we will fetch a series of blocks using the rosetta-sdk-go. The
rosetta-sdk-go contains a client that was generated from the rosetta-specifications
repository.

Create Go File

To use the rosetta-sdk-go, we create a file called main.go and add the following contents:

package main

import (
	"context"
	"log"

	"github.com/coinbase/rosetta-sdk-go/fetcher"
	"github.com/coinbase/rosetta-sdk-go/types"
)

const (
	// serverURL is the URL of the running Celo Data API implementation.
	serverURL = "http://localhost:8080"
)

func main() {
	ctx := context.Background()

	// Step 1: Create a new fetcher
	newFetcher := fetcher.New(
		serverURL,
	)

	// Step 2: Initialize the fetcher's asserter
	//
	// Behind the scenes this makes a call to get the
	// network status and uses the response to inform
	// the asserter what are valid responses.
	primaryNetwork, networkStatus, err := newFetcher.InitializeAsserter(ctx)
	if err != nil {
		log.Fatal(err)
	}

	// Step 3: Print the primary network and network status
	log.Printf("Primary Network: %s\n", types.PrettyPrintStruct(primaryNetwork))
	log.Printf("Network Status: %s\n", types.PrettyPrintStruct(networkStatus))

	// Step 4: Fetch the current block with retries (automatically
	// asserted for correctness)
	block, err := newFetcher.BlockRetry(
		ctx,
		primaryNetwork,
		types.ConstructPartialBlockIdentifier(
			networkStatus.CurrentBlockIdentifier,
		),
	)
	if err != nil {
		log.Fatal(err)
	}

	// Step 5: Print the block
	log.Printf("Current Block: %s\n", types.PrettyPrintStruct(block))

	// Step 6: Get a range of blocks (genesis -> genesis + 10)
	blockMap, err := newFetcher.BlockRange(
		ctx,
		primaryNetwork,
		networkStatus.GenesisBlockIdentifier.Index,
		networkStatus.GenesisBlockIdentifier.Index+10,
	)
	if err != nil {
		log.Fatal(err)
	}

	// Step 7: Print the block range
	log.Printf("Block Range: %s\n", types.PrettyPrintStruct(blockMap))
}

Fetch First 10 Blocks

After creating the file, we run it with the following command:

go run main.go

Running the command will print the current network status (i.e. the
syncing progress), the current block contents, and the block range from
0-9.

Primary Network: {
 "blockchain": "celo",
 "network": "42220"
}
Network Status: {
 "current_block_identifier": {
  "index": 939934,
  "hash": "0x6aec639c35225d386a215f6c581892478b9d2fe67ac69d0fdbb7f0e461bbc60b"
 },
 "current_block_timestamp": 1592273720000,
 "genesis_block_identifier": {
  "index": 0,
  "hash": "0x19ea3339d3c8cda97235bc8293240d5b9dadcdfbb5d4b0b90ee731cac1bd11c3"
 },
 "peers": [
  {
   "peer_id": "00189bd5b6019cd839a6dc2aa54703fec7ee25b1d6700250cb6a1fb45b025b2f"
  }
 ]
}
Current Block: {
 "block_identifier": {
  "index": 939934,
  "hash": "0x6aec639c35225d386a215f6c581892478b9d2fe67ac69d0fdbb7f0e461bbc60b"
 },
 "parent_block_identifier": {
  "index": 939933,
  "hash": "0xb6e1d3021e0da65eb82e198db95e1749344f5ef23f7097628c8a559057e5884b"
 },
 "timestamp": 1592273720000,
 "transactions": null
}
Block Range: {
 "0": {
  "block_identifier": {
   "index": 0,
   "hash": "0x19ea3339d3c8cda97235bc8293240d5b9dadcdfbb5d4b0b90ee731cac1bd11c3"
  },
  "parent_block_identifier": {
   "index": 0,
   "hash": "0x19ea3339d3c8cda97235bc8293240d5b9dadcdfbb5d4b0b90ee731cac1bd11c3"
  },
  "timestamp": 1587571200000,
  "transactions": [
   {
    "transaction_identifier": {
     "hash": "0x19ea3339d3c8cda97235bc8293240d5b9dadcdfbb5d4b0b90ee731cac1bd11c3"
    },
    "operations": []
   }
  ]
 },
 "1": {
  "block_identifier": {
   "index": 1,
   "hash": "0x805a4ceaec7435dc38a74eb036b99929953d7dd14af218c8a19430d52ca1ac57"
  },
  "parent_block_identifier": {
   "index": 0,
   "hash": "0x19ea3339d3c8cda97235bc8293240d5b9dadcdfbb5d4b0b90ee731cac1bd11c3"
  },
  "timestamp": 1587571205000,
  "transactions": null
 },
 "10": {
  "block_identifier": {
   "index": 10,
   "hash": "0x5ffe9b7813f766e33b1e0c4991cd9e25527eb3fe76417b1b4cbdfb77a797e8c8"
  },
  "parent_block_identifier": {
   "index": 9,
   "hash": "0x4b1fb544c335c206b7738a5a38c4a81f1d1f1589c3d78e53d040d78330676aa8"
  },
  "timestamp": 1587571787000,
  "transactions": null
 },
 "2": {
  "block_identifier": {
   "index": 2,
   "hash": "0x2c87485b84b02a21e573faa581c6e1f2b677fdd39ca184095e125fa510c38809"
  },
  "parent_block_identifier": {
   "index": 1,
   "hash": "0x805a4ceaec7435dc38a74eb036b99929953d7dd14af218c8a19430d52ca1ac57"
  },
  "timestamp": 1587571747000,
  "transactions": null
 },
 "3": {
  "block_identifier": {
   "index": 3,
   "hash": "0xa26a855e33fd6b9ffaf9512b944b6cc92209b09398d31f80f047e71f11000000"
  },
  "parent_block_identifier": {
   "index": 2,
   "hash": "0x2c87485b84b02a21e573faa581c6e1f2b677fdd39ca184095e125fa510c38809"
  },
  "timestamp": 1587571752000,
  "transactions": null
 },
 "4": {
  "block_identifier": {
   "index": 4,
   "hash": "0xa220ba5e0619e6fdcb931be62e92097d478a140da79b43ad509511df15683e4c"
  },
  "parent_block_identifier": {
   "index": 3,
   "hash": "0xa26a855e33fd6b9ffaf9512b944b6cc92209b09398d31f80f047e71f11000000"
  },
  "timestamp": 1587571757000,
  "transactions": null
 },
 "5": {
  "block_identifier": {
   "index": 5,
   "hash": "0x689bb742686b69c5fa9c0c2eaea5a0e3f4a3afd175ea5d57dc78b5d1917cc382"
  },
  "parent_block_identifier": {
   "index": 4,
   "hash": "0xa220ba5e0619e6fdcb931be62e92097d478a140da79b43ad509511df15683e4c"
  },
  "timestamp": 1587571762000,
  "transactions": null
 },
 "6": {
  "block_identifier": {
   "index": 6,
   "hash": "0xfec7de9347ff200eca4a5445525d39f39d934b17153c657bad3f8323f0d2c2c0"
  },
  "parent_block_identifier": {
   "index": 5,
   "hash": "0x689bb742686b69c5fa9c0c2eaea5a0e3f4a3afd175ea5d57dc78b5d1917cc382"
  },
  "timestamp": 1587571767000,
  "transactions": null
 },
 "7": {
  "block_identifier": {
   "index": 7,
   "hash": "0x119174032577c6cd4db213a4c0d459bdf89fa1d19361d9246222b72bebd091ba"
  },
  "parent_block_identifier": {
   "index": 6,
   "hash": "0xfec7de9347ff200eca4a5445525d39f39d934b17153c657bad3f8323f0d2c2c0"
  },
  "timestamp": 1587571772000,
  "transactions": null
 },
 "8": {
  "block_identifier": {
   "index": 8,
   "hash": "0x80d982985bcc958ac4d83595619e1d5d6f4581640c6c5ca38af236d88b89c1f2"
  },
  "parent_block_identifier": {
   "index": 7,
   "hash": "0x119174032577c6cd4db213a4c0d459bdf89fa1d19361d9246222b72bebd091ba"
  },
  "timestamp": 1587571777000,
  "transactions": null
 },
 "9": {
  "block_identifier": {
   "index": 9,
   "hash": "0x4b1fb544c335c206b7738a5a38c4a81f1d1f1589c3d78e53d040d78330676aa8"
  },
  "parent_block_identifier": {
   "index": 8,
   "hash": "0x80d982985bcc958ac4d83595619e1d5d6f4581640c6c5ca38af236d88b89c1f2"
  },
  "timestamp": 1587571782000,
  "transactions": null
 }
}

Extra Credit

Now that you’ve fetched a few blocks from a running Data API implementation, try building a basic block
explorer by extending the rosetta-sdk-go example above.

If you make anything awesome, please reach out to rosetta@coinbase.com - we may feature it on our blog!


This website contains links to third-party websites or other content for information purposes only (“Third-Party Sites”). The Third-Party Sites are not under the control of Coinbase, Inc., and its affiliates (“Coinbase”), and Coinbase is not responsible for the content of any Third-Party Site, including without limitation any link contained in a Third-Party Site, or any changes or updates to a Third-Party Site. Coinbase is not responsible for webcasting or any other form of transmission received from any Third-Party Site. Coinbase is providing these links to you only as a convenience, and the inclusion of any link does not imply endorsement, approval or recommendation by Coinbase of the site or any association with its operators.

Implementing the Rosetta tools and/or guidance found on the Rosetta website does not guarantee an asset to be listed on Coinbase. Coinbase evaluates prospective assets against our Digital Asset Framework to assess factors like security, compliance, and the project’s alignment with our mission of creating an open financial system for the world. To apply for listing, fill out an application here.

Coinbase does not endorse, promote, or specify the listing or integration requirements for any third party projects, exchanges, or cryptocurrencies mentioned in this blogpost or otherwise. Any descriptions of functionality and services provided are for information only. Coinbase is not responsible for any loss of funds or other damages caused as a result of using the projects described above.

Unless otherwise noted, all images provided herein are by Coinbase.