I was curious how people approach this. For example, we were thinking about saving transactions to a nosql db with the transaction hash as the primary key and block identifier. Granted, this is a gigantic data set. Was curious what community recommendations would be, alternative ways to approach it, or hear feedback from anyone who has tried this out before. Thanks!
An open-source implementation of an indexer like this is probably the number 1 item on our wishlist. We’ve been working on something internally that uses many of the rosetta-sdk-go components to reduce the amount of new code that needs to be written (like the syncer).
It sounds like you are planning to store this in some sort of cloud-provider DB? If not, the storage package already exposes many of the endpoints you may want to access (it also deals with duplicate tx hashes). For example, here is the “transaction by hash” endpoint:
In our own research, the biggest complication is usually just handling orphan blocks. It can lead to huge rollbacks of data if not approached carefully (and make writing /account/transaction queries complicated).
I saw @Coinbox.org pop-up a few times on here. Was wondering if you guys had any projects that could help with this?
Hey, @stuart! Just wanted to give you a heads up that we are about to merge support for standard "indexer " endpoints in the Rosetta Spec. We anticipate that many people will require functionality like what you’ve described on this issue and we want to avoid a proliferation of interfaces/access patterns built to access this kind of data. If you decide to create an indexer, hopefully you’ll consider supporting these endpoints so anyone using Rosetta can access it without additional configuration or work!