Multiple nodes Tx was not accepted to mempool setting INVALID flag errors

Question

Dear Multichain team,

Firstly I LOVE your product, it is truly awesome. Everything works perfectly and as described in the documentation, except for one issue that's bugging me in the last couple of days.

The situation is: I have 2 nodes, which both are set to mine, with mining diversity 0 (per my proof of concept requirements). All works really well when the chain is under normal load i.e. few transactions, But the moment I load 60 transactions per min, I keep on getting the below errors:

2016-11-01 12:42:20 Tx ff7ccbf7144672c8a5c1b120d111f14fc038528f196be0da012daba96b760954 was not accepted to mempool, setting INVALID flag

2016-11-01 12:42:20 Missing tx (6dc6560bcfe12695cfd1cf51be0e8dd4c207133653e47c0e5397f50886888d9f)

2016-11-01 12:42:20 Tx ff97555d34508942fabf6e6b2616011db90fc06d9f575607370140ea00f57b29 was not accepted to mempool, setting INVALID flag

2016-11-01 12:42:20 Missing tx (bff45a2d27171535017430ed26fbaf5533d6da151f3324e5f215833e768c2188)

2016-11-01 12:42:20 Tx ffa227f215f733a0180f3860735d114fb24ae37a421d95a5139a561557947222 was not accepted to mempool, setting INVALID flag

2016-11-01 12:42:20 Missing tx (5f3021fd912c98b06b8a27ed211ba25d9c01b3760594d9321a5c4181c09399b1)

Looking at the logs (happy to zip and send them to you), it seems as there are forks happening because both the nodes are mining, and a large number of transactions are falling in this state. Looking at your other threads, if there is a forking problem, then these transactions are expected to rejoin the mempool right, but somehow many transactions with transaction IDs are never getting mined and keep throwing these errors.

Could you please give me some clues to resolve this issue.

My transactions are pretty normal one, just transfer of 1 xcoin from address A to B, with some metadata stored in it. And these errors are never happening when I run just a single node.

Thanks in advance.

Comments

Thanks for reporting this - we'd like to take a look. Please stop both nodes, zip up their blockchain directories, and then send to multichain dot debug at gmail dot com. Thanks!

---

Done. Created a brand new set of nodes and replicated the issue. Sent all the files and an example transaction that failed. Please let me know if you need any more data. Thanks again for your help.

Answer 1

It looks like normal behaviour as you are trying to make double-spending (I'm not sure how did you do it, but probably you imported private key from one node to another):

Node 1:

2016-11-02 04:53:52 CommitTransaction:
CTransaction(hash=c4467c4040, ver=1, vin.size=1, vout.size=3, nLockTime=0)
    CTxIn(COutPoint(0d7def897f, 2), scriptSig=3045022100d84ff4701b82bf)
    CTxOut(nValue=0.00000000, scriptPubKey=OP_DUP OP_HASH160 219eb26b3050)
    CTxOut(nValue=0.00000000, scriptPubKey=OP_RETURN 7b226c6f616e4170706c)
    CTxOut(nValue=0.00000000, scriptPubKey=OP_DUP OP_HASH160 a71b2f7f9a00)

Node 2:

2016-11-02 04:53:52 CommitTransaction:
CTransaction(hash=9d6682478e, ver=1, vin.size=1, vout.size=3, nLockTime=0)
    CTxIn(COutPoint(0d7def897f, 2), scriptSig=3045022100abed8b598338a8)
    CTxOut(nValue=0.00000000, scriptPubKey=OP_DUP OP_HASH160 219eb26b3050)
    CTxOut(nValue=0.00000000, scriptPubKey=OP_RETURN 7b226c6f616e4170706c)
    CTxOut(nValue=0.00000000, scriptPubKey=OP_DUP OP_HASH160 a71b2f7f9a00)

Please note that you are spending the same input COutPoint(0d7def897f, 2),

This normally lead to :

Node 1:

2016-11-02 04:53:52 Conflicting with in-memory CTxIn(COutPoint(0d7def897f, 2), scriptSig=3045022100abed8b598338a8)
2016-11-02 04:53:52 Missing tx (9d6682478efc9eb15bf912810caf92de65188def0b6380215e649d57b4952b29)
 

Node 2:

2016-11-02 04:53:52 Conflicting with in-memory CTxIn(COutPoint(0d7def897f, 2), scriptSig=3045022100d84ff4701b82bf)
2016-11-02 04:53:52 Missing tx (c4467c404057bbdb68c592e74831bea0952f5b8838a676fe883fa4e7ef9d640a)

 

On each node only one transaction is valid. But... you continue to create transactions on each node separately using these two transactions, so the number of conflicting transactions grows.

Now. You are playing with blockchain reorganizations (by switching on/off mining on each node). As a result you can get two long conflicting chains. And every time you switch the chain, all transactions on alternative chain become INVALID.

Next. As private key is shared between wallets, both wallets store all transactions, valid and invalid. From time to time wallet tries to re-accept invalid transaction (as they may become valid because of blockchain reorganization). If the attempt fails, you also see this error message. We will optimize this process in the future.

It is natural behaviour in case of double-spending. To avoid this you should not share private keys between nodes.

Comments

Hi Michael,

Perfect response and spot on. I've been using the same from address (and their private keys) in both the nodes. Thanks for the clarification, much appreciated.

A quick question to redesign my transactions - if I use sendtoaddress API, which from address is picked by default? Is it possible to specify getaccounts[0] as the from address in sendassetfrom instead of a physical address.

Thanks,Naveen.

---

When using sendtoaddress all unspent outputs belonging to all addresses in the wallet can be used, the choice depend on several factors, any address (or even several) can be chosen. When using sendfromaddress only outputs from specified address are used.

You can get list of addresses using getaddresses API and list of unspent outputs using listunspent. Accounts mechanism is to be deprecated in the future.

But you can continue using sendtoaddress - just don't use importprivkey or importwallet.  In this case wallets (nodes) will have separate sets of addresses and you will never double spend.

---

Understood! Thanks again, much appreciated.

Once again, multichain is awesome! Great work