Operation.Status must be empty for construction

nguyennguyen · December 3, 2020, 7:48am

Detail log from cli

 REQUEST /construction/preprocess network_identifier:{"blockchain":"TomoChain","network":"1992"} intent:[{"operation_identifier":{"index":0},"type":"CALL","status":"","account":{"address":"0xc70f010E8DB8bc436712A93D170C7c27Db1981Ea"},"amount":{"value":"-168631456366628516104002996126486808","currency":{"symbol":"TOMO","decimals":18}}},{"operation_identifier":{"index":1},"type":"CALL","status":"","account":{"address":"0xd612630827E53CD4164032CA9c7dd12A248D3f51"},"amount":{"value":"168631456366628516104002996126486808","currency":{"symbol":"TOMO","decimals":18}}}] metadata:null



2020/12/03 14:39:05 error {"code":0,"message":"Operation.Status must be empty for construction: operation status is invalid in operation 0","retriable":false} assertion failed: Error.Code unexpected: code 0
2020/12/03 14:39:05 ERROR /construction/preprocess error:{"err":{},"client_err":{"code":0,"message":"Operation.Status must be empty for construction: operation status is invalid in operation 0","retriable":false}}

I used this configuration

github.com

coinbase/rosetta-ethereum/blob/master/rosetta-cli-conf/testnet/ethereum.ros

request_funds(1){
  find_account{
    currency = {"symbol":"ETH", "decimals":18};
    random_account = find_balance({
      "minimum_balance":{
        "value": "0",
        "currency": {{currency}}
      },
      "create_limit":1
    });
  },

  // Create a separate scenario to request funds so that
  // the address we are using to request funds does not
  // get rolled back if funds do not yet exist.
  request{
    loaded_account = find_balance({
      "account_identifier": {{random_account.account_identifier}},
      "minimum_balance":{
        "value": "10000000000000000",

This file has been truncated. show original

I didn’t put any status in construction check scenarios ethereum.ros, but /construction/preprocess got status=""

Could you please help to take a look ?

I got this error after upgrading to github.com/coinbase/rosetta-sdk-go v0.6.1
It seemed to work before when I use github.com/coinbase/rosetta-sdk-go v0.3.4

patrick.ogrady · December 3, 2020, 5:01pm

Sorry for the confusion here, @nguyennguyen! This is related to our introduction of a backwards-incompatible change in rosetta-sdk-go. We’ve since fixed the issue:

Let me know if you are still having issues after updating to rosetta-sdk-go@v0.6.3 and rosetta-cli@v0.6.2!

nguyennguyen · December 4, 2020, 4:27am

Thank you @patrick.ogrady . It works with rosetta-sdk-go v0.6.3

nguyennguyen · December 4, 2020, 10:15am

Hi @patrick.ogrady
check:construction works well
But rosetta-cli running check:data crash with this error


panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x20 pc=0x9ae7de]

goroutine 72 [running]:
github.com/coinbase/rosetta-sdk-go/storage/modules.(*BalanceStorage).EstimatedReconciliationCoverage(0xc0000d2140, 0xe5e0e0, 0xc0000af740, 0x0, 0x0, 0x0)
	/go/pkg/mod/github.com/coinbase/rosetta-sdk-go@v0.6.3/storage/modules/balance_storage.go:458 +0xde
github.com/coinbase/rosetta-cli/pkg/results.ComputeCheckDataStats(0xe5e0e0, 0xc0000af740, 0xc00000e5a0, 0xc0000d2140, 0xc04865ff40)
	/source/pkg/results/data_results.go:253 +0x608
github.com/coinbase/rosetta-cli/pkg/results.ComputeCheckDataStatus(0xe5e0e0, 0xc0000af740, 0xc0000c81b0, 0xc00000e5a0, 0xc0000d2140, 0xc0000ae0c0, 0xc0004b0bd0, 0xc00007d040, 0x0)
	/source/pkg/results/data_results.go:387 +0x50
github.com/coinbase/rosetta-cli/pkg/tester.(*DataTester).StartPeriodicLogger(0xc0000c2370, 0xe5e0e0, 0xc0000af740, 0x0, 0x0)
	/source/pkg/tester/data.go:446 +0x165
github.com/coinbase/rosetta-cli/cmd.runCheckDataCmd.func1(0x0, 0x0)
	/source/cmd/check_data.go:126 +0x3c
golang.org/x/sync/errgroup.(*Group).Go.func1(0xc0000c9260, 0xc00000ee00)
	/go/pkg/mod/golang.org/x/sync@v0.0.0-20200625203802-6e8e738ad208/errgroup/errgroup.go:57 +0x59
created by golang.org/x/sync/errgroup.(*Group).Go
	/go/pkg/mod/golang.org/x/sync@v0.0.0-20200625203802-6e8e738ad208/errgroup/errgroup.go:54 +0x66
➜  bin git:(master) ✗    5:03:44                                        ./rosetta-cli version
v0.6.2

I was running check:data with this configuration

    "data": {
        "active_reconciliation_concurrency": 0,
        "inactive_reconciliation_concurrency": 0,
        "inactive_reconciliation_frequency": 0,
        "log_blocks": false,
        "log_transactions": false,
        "log_balance_changes": false,
        "log_reconciliations": false,
        "ignore_reconciliation_error": false,
        "exempt_accounts": "",
        "bootstrap_balances": "",
        "interesting_accounts": "",
        "reconciliation_disabled": false,
        "inactive_discrepency_search_disabled": false,
        "balance_tracking_disabled": true,
        "coin_tracking_disabled": false,
        "start_index": 120,
        "results_output_file": ""
    }

Could you please help to take a look ?
Thank in advance !

nguyennguyen · December 7, 2020, 4:12am

Hi @patrick.ogrady
It seems something went wrong with my reconciliation configuration
Setting "reconciliation_disabled": true, it works
So, could you guide me about reconciliation and how to set up its configuration + bootstrap_balance to pass this test ?

patrick.ogrady · December 9, 2020, 3:28am

@nguyennguyen this panic is due to your configuration avoiding a sanity check in rosetta-cli. We will make sure to add an assertion in the next release.

TL;DR you should not have reconciliation_disabled=false and balance_tracking_disabled=true. I’m guessing you mean to do (set balance_tracking_disabled=false and remove start_index):

"data": {
        "active_reconciliation_concurrency": 0,
        "inactive_reconciliation_concurrency": 0,
        "inactive_reconciliation_frequency": 0,
        "log_blocks": false,
        "log_transactions": false,
        "log_balance_changes": false,
        "log_reconciliations": false,
        "ignore_reconciliation_error": false,
        "exempt_accounts": "",
        "bootstrap_balances": "",
        "interesting_accounts": "",
        "reconciliation_disabled": false,
        "inactive_discrepency_search_disabled": false,
        "balance_tracking_disabled": false,
        "coin_tracking_disabled": false,
        "results_output_file": ""
    }

rosetta-cli performs 2 types of reconciliation (active and inactive). You can read more about what that means here:

Often times, bootstrap_balances isn’t necessary for blockchains that implement historical balance lookup (unless there are balances set in a genesis file). For those without context, bootstrapping balance is necessary to set the computed balance of some number of accounts to non-zero balance at genesis. If this is not done, it will not be possible to compare rosetta-cli's computed balance during reconciliation.

In rosetta-bitcoin, we don’t need to bootstrap any balances because there are no balances prior to the genesis block. However, we do need to bootstrap balances for rosetta-ethereum because of all the accounts credited from the crowdsale:

github.com

coinbase/rosetta-ethereum/blob/master/rosetta-cli-conf/mainnet/bootstrap_balances.json

[
 {
  "account_identifier": {
   "address": "0x000D836201318Ec6899a67540690382780743280"
  },
  "currency": {
   "symbol": "ETH",
   "decimals": 18
  },
  "value": "200000000000000000000"
 },
 {
  "account_identifier": {
   "address": "0x001762430ea9C3A26e5749afdB70da5F78DDBB8C"
  },
  "currency": {
   "symbol": "ETH",
   "decimals": 18
  },
  "value": "200000000000000000000"

This file has been truncated. show original

Let me know if you have any follow-ups (if you do, please create a new topic as this one is getting a little off topic)!

nguyennguyen · December 9, 2020, 4:22am

Copied this configuration. Looks good so far with empty bootstrap_balance
But our blockchain initialized some balances at genesis block. I’d like to set them in bootstrap_balances.json

github.com

tomochain/tomochain-rosetta-gateway/blob/trace-transaction/rosetta-cli-conf/bootstrap_balances.json

[
 {
  "account_identifier": {
   "address": "0x0000000000000000000000000000000000000068"
  },
  "currency": {
   "symbol": "TOMO",
   "decimals": 18
  },
  "value": "16000000000000000000000000"
 },
  {
  "account_identifier": {
   "address": "0x0000000000000000000000000000000000000088"
  },
  "currency": {
   "symbol": "TOMO",
   "decimals": 18
  },
  "value": "150000000000000000000000"

This file has been truncated. show original

Not sure whether I miss any implementation for this

2020/12/09 11:18:12 Setting account 0x0000000000000000000000000000000000000068 balance to 16000000000000000000000000 &{Symbol:TOMO Decimals:18 Metadata:map[]}
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x20 pc=0x9ad979]

goroutine 1 [running]:
github.com/coinbase/rosetta-sdk-go/storage/modules.(*BalanceStorage).SetBalance(0xc00007a230, 0xe5e0e0, 0xc0000200c0, 0xe60980, 0xc0000c2480, 0xc00000e600, 0xc0004ab680, 0xc00000ec20, 0xe5e001, 0xc0004ab698)
	/go/pkg/mod/github.com/coinbase/rosetta-sdk-go@v0.6.3/storage/modules/balance_storage.go:358 +0xb9
github.com/coinbase/rosetta-sdk-go/storage/modules.(*BalanceStorage).BootstrapBalances(0xc00007a230, 0xe5e0e0, 0xc0000200c0, 0xc000027cc0, 0x17, 0xc00000ec20, 0x0, 0x0)
	/go/pkg/mod/github.com/coinbase/rosetta-sdk-go@v0.6.3/storage/modules/balance_storage.go:1115 +0x3fd
github.com/coinbase/rosetta-cli/pkg/tester.InitializeData(0xe5e0e0, 0xc0000200c0, 0xc000482200, 0xc0004b0bd0, 0xc000020180, 0xc0004c6600, 0xc00000ec20, 0x0, 0x14515c9, 0x7000106)
	/source/pkg/tester/data.go:190 +0x18b0

patrick.ogrady · December 9, 2020, 4:33am

nguyennguyen:

Not sure whether I miss any implementation for this

2020/12/09 11:18:12 Setting account 0x0000000000000000000000000000000000000068 balance to 16000000000000000000000000 &{Symbol:TOMO Decimals:18 Metadata:map[]}
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x20 pc=0x9ad979]

goroutine 1 [running]:
github.com/coinbase/rosetta-sdk-go/storage/modules.(*BalanceStorage).SetBalance(0xc00007a230, 0xe5e0e0, 0xc0000200c0, 0xe60980, 0xc0000c2480, 0xc00000e600, 0xc0004ab680, 0xc00000ec20, 0xe5e001, 0xc0004ab698)
	/go/pkg/mod/github.com/coinbase/rosetta-sdk-go@v0.6.3/storage/modules/balance_storage.go:358 +0xb9
github.com/coinbase/rosetta-sdk-go/storage/modules.(*BalanceStorage).BootstrapBalances(0xc00007a230, 0xe5e0e0, 0xc0000200c0, 0xc000027cc0, 0x17, 0xc00000ec20, 0x0, 0x0)
	/go/pkg/mod/github.com/coinbase/rosetta-sdk-go@v0.6.3/storage/modules/balance_storage.go:1115 +0x3fd
github.com/coinbase/rosetta-cli/pkg/tester.InitializeData(0xe5e0e0, 0xc0000200c0, 0xc000482200, 0xc0004b0bd0, 0xc000020180, 0xc0004c6600, 0xc00000ec20, 0x0, 0x14515c9, 0x7000106)
	/source/pkg/tester/data.go:190 +0x18b0

Ahhhh you found another regression related to an earlier optimization we made . We are bootstrapping balances before fully initializing storage. I’ll queue this for the next release! Thanks for your patience!

nguyennguyen · December 9, 2020, 4:35am

ok @patrick.ogrady
I’ll let boostrap_balance empty now and wait for the next release

Thank you so much !

patrick.ogrady · December 9, 2020, 5:09pm

Just merged the fix in rosetta-cli!

Will ping this thread when the next release is out!

patrick.ogrady · December 9, 2020, 8:35pm

Here you go!

nguyennguyen · December 10, 2020, 10:54am

Thank you Patrick. It works
By the way, Must balance_tracking and reconciliation be enable for all blockchains ?

patrick.ogrady · December 10, 2020, 3:02pm

We only consider an implementation’s Data API “Rosetta-compliant” once it passes rosetta-cli check:data with balance_tracking and reconciliation enabled.

That being said, other integrators that don’t care about full chain data fidelity (capturing all balance-changing events) can use your implementation before it passes all tests if they want!