Traversing by block index problem

We are having an issue when running the rosetta-cli check command and we think it’s related on how Cardano and the cli work together. I’ll bullet some points as it’s easier to read:

  • The error we are getting is Check failed: cannot remove genesis block: unable to sync to 999 although it seems Genesis block was properly parsed [STATS] Blocks: 1 (Orphaned: 0) Transactions: 14505 Operations: 14505 Reconciliations: 14445 (Inactive: 0)
  • As far as we understand, you are traversing the blockchain by block number (which makes sense unless you want to go from the tip backwards or add another field like next block but it will force you to process sequentially instead of in parallel)
  • In Cardano blockchain there are some special blocks which are created between epochs. For example: bda92f619fa9ff2c486c493bfa19c061a40f66a1901db30c8651478d5982ab27 or 89d9b5a5b8ddc8d7e5a6795e9774d97faf1efea59b2caf7eaf9f8c5b32059df4 . You can search them by hash using this explorer https://explorer.cardano.org/en but I’m attaching an example.
  • Such blocks don’t have neither block number nor slot number.
  • The blockchain looks like Genesis (0) <-- EBB <-- Block (1 )<-- Block (2) <-- .....<-- Block (n-1) <-- EBB <-- Block (n)
  • Our understanding is that this check 1 is failing because Block 1 parent is EBB and not Genesis but the CLI has never retrieved it. Are we getting it right? If so:
  1. Do you think this is a specific issue when traversing the blockchain by number? Do you do it like this in your internal systems?
  2. How should we proceed to be compliant with Rosetta API?

Thanks!

Your debugging into the source of this error is exactly right, @alan.verbner.

Because Block (1) is not connected to Block (0)-Genesis, the rosetta-cli thinks that Block (0) has been orphaned and that the EBB is its new parent. In Rosetta, it is assumed that there is some canonical, connected chain of blocks and that each block in this chain has a unique index. If you want to open a PR on the spec to reflect this, I would be happy to review it! Here is an example:

Follow-Up

  1. We do not support “index-less” blocks in our internal systems. Our requirements across all our systems map directly to the requirements in the Rosetta API and our syncing requirements map to the behavior of the rosetta-cli. We very purposely did this to catch nuances like this at the testing phase (which it appears we did here).
  2. Before I answer this question, I was wondering if these epoch boundary blocks ever contain any balance-changing Operations or even Transactions for that matter (the samples I looked at did not contain any)? I also heard they may be removed at the Shelley hard fork (not that this means we won’t have to parse them to support pre-Shelley blocks)?

Hi @patrick.ogrady… yeah, EBB are not something “interesting” as they don’t have any operations although there are there and linked.

What we have done so far is to skip such blocks and now rosetta-cli works as expected. Obviously, if you try to try traverse the blockchain by index (frm the tip backwards) you will see different blocks although the total balance and operations will be the same.

I think this example highlights one of the more opinionated aspects of the Rosetta abstraction. It was designed to make the extraction of all balance-changing events from a blockchain straightforward (staking, transfers, smart contract interaction, etc. ) but it was not a goal to capture more nuanced, non-balance-changing events with high fidelity.

Suggestion

My recommendation here is to skip epoch boundary blocks when linking blocks together (i.e. abstract them away in your implementation). I don’t question that it is useful for your protocol but they aren’t necessary (and even prevent) Rosetta syncing. In case some users want to view these EBBs, I recommend including their hash in the Block.Metadata of the first block of an epoch.

Here is an example illustrating the suggestion:

block 0 -> EBB
    \----------> block 1 (Metadata: {boundary: EBB}) -> block 2

Let me know if you have any additional questions!

Thanks @patrick.ogrady we have already decided to skip them and everything is looking good so far.

1 Like