How to prevent some UTXOs to be selected during transaction creation process

Hello,

We are implementing rosetta support for the Horizen blockchain (https://github.com/HorizenOfficial/zen). We forked rosetta-bitcoin (https://github.com/HorizenOfficial/rosetta-zen) and adapted for our particular needs (e.g. replay protection, shielded transactions).

The last doubt remaining is the management of coinbase transactions (miner reward). In particular our consensus requires using shielded transactions to spend such coins. We are worried that if we don’t mark somehow this coinbase UTXOs they can be selected during the transaction creation process (e.g. account/balance ?).

Thank you in advance!

1 Like

:wave: @Marco

Very cool! Glad you found it useful! Make sure to checkout the upgrade we just released to provide historical balance tracking:

The easiest way to “hide” these UTXOs (if you are only providing support for unshielded transactions in your implementation) is to not populate the CoinChange field on the output of “coinbase” transactions:

This will effectively make those UTXOs invisible to Rosetta but will still allow for balance tracking (which will be important for passing rosetta-cli check:data).

Hope that helps!

1 Like

You may wish to hide this feature behind a runtime flag so that integrators that want this visibility (for analytics or something) can still access it. If that makes sense?

1 Like

Hi,

Thank you for you answer!

I tried to not populate the CoinChange field but when Rosetta tries to sync a block containing a shielded transaction, that tries to spend one input of this type, it stops to work. I guess the problem is that it doesn’t find the input of the shielded transaction inside its storage.

We found another solution that appends a special character at the end of coinbase outputs addresses, with this method we index those coins to a non existing address in the database and they will never be selected for transactions.

Do you think this could be a good solution?

Thanks.

1 Like

Ahhh, yes! I forgot about that check. If you ensure those coins are never spent in the corresponding INPUT operation (leave CoinChange blank for any shielded inputs but keep the address and amount), it would work (you can still add CoinActions for any new coins that are created in OUTPUT operations).

I usually advise against creating “synthetic/hallucinated” data for the purposes of deterring certain behavior. I wrote a little about this here:

Could you elaborate on what you mean by “index those coins to a non existing address in the database and they will never be selected for transactions”? Do you mean you are just avoiding returning these coins in /account/balance?

To be honest, I think just including these special coinbases and rejecting them during /construction/metadata (online call where you can check the scriptpub key) as unusable “coins” may be the cleanest/easiest? Then anyone using your implementation for analytics/data can still view these coins? I think putting this burden on the user to know these must be spent with a shielded tx is fine (and then at some point you could add shielded support in your Construction API implementation without breaking your Data API implementation).

Maybe we should add a metadata field to Coin so you can communicate this with must_spend_shielded or something?

2 Likes

I’d highly recommend rebasing onto our latest release btw…greatly improved the sync speed!

2 Likes

Ahhh, yes! I forgot about that check. If you ensure those coins are never spent in the corresponding INPUT operation (leave CoinChange blank for any shielded inputs but keep the address and amount), it would work (you can still add CoinActions for any new coins that are created in OUTPUT operations).

Would this approach imply leaving the coinbase UTXOs forever unspent and never being deleted from the database? Moreover if I understand correctly we don’t validate the existence of all the inputs even if not coinbase. Is it correct?

I usually advise against creating “synthetic/hallucinated” data for the purposes of deterring certain behavior. I wrote a little about this here:

I see what you mean, the idea was to append a string like “_coinbase” to the address for coinbase outputs. A sort of subaccount that would make the UTXOs still available for spending with shielded inputs but automatically not listed in the account/balance.
For our understanding this has the side effect of having to look also for “_coinbase” inputs in case in the future we are going to implement shielded transactions in Rosetta.
Alternatively, if we understood correctly the SubAccount purpose, a more elegant solution would be to index the coinbase outputs using the original address but populating also the SubAccountIdentifier field (like SubAccount.Address = “coinbase”). This should make these coins available to be spent but not listed in the account/balance endpoint for the main address. How do you see it?

To be honest, I think just including these special coinbases and rejecting them during /construction/metadata (online call where you can check the scriptpub key) as unusable “coins” may be the cleanest/easiest? Then anyone using your implementation for analytics/data can still view these coins? I think putting this burden on the user to know these must be spent with a shielded tx is fine (and then at some point you could add shielded support in your Construction API implementation without breaking your Data API implementation).

I have a doubt about this, but maybe i miss something.
What happens if it’s needed to create a transaction but the account/balance endpoint keep returning the same list of UTXOs also including the unspendable coinbase outputs? I understand the construction/metadata endpoint will reject such UTXOs combination, but if the same list is provided for additional attempts, the whole address’ coins will be stuck. Am I right?

I’d highly recommend rebasing onto our latest release btw…greatly improved the sync speed!

Yes thank you, we are also working to integrating the latest changes.

Thank you!

This approach would basically be treating them as if they never existed. They wouldn’t be created when the block is mined and wouldn’t be destroyed when spent. Just to over-communicate, the balance of these would still be tracked (using Operation.Amount) but the Coin itself would just never be represented in Operations.

You may need to still store these Coins in the database to return the right balance for /account/balance but you wouldn’t add them to Operations in returned blocks.

As you can see, this is getting a little confusing with this approach haha…

Anything you don’t include in a Operation.CoinAction will not be validated. If you have a Operation.CoinAction that creates a coin but there is never a corresponding Operation.CoinAction that spends that coin, it will exist indefinitely. In the opposite case (which you ran into in an earlier message), it will return an error because a non-existent coin cannot be spent.

Oooo this is a very interesting approach. I like this A LOT! It would then be possible to see all data without accidentally trying to spend coins that have certain conditions (without trying to parse some sort of metadata).

Just to make sure I understand, you’d represent balance as the following (assuming addr1 has a balance of 15 that can be spent without shielded and a balance of 10 that must be spent with shielded)?:

/address/balance {address: "addr1"} = 15
/address/balance {address: "addr1", sub_account: {address: "coinbase"}} = 10

Total Balance of "addr1" = 25

Let’s just disregard this line of exploration. I think your SubAccount idea is much better. You are correct that the /account/balance endpoint would keep returning coins that couldn’t be spent which may be pretty confusing!

1 Like

Exactly!
Thank you again for your help!

1 Like