-
Notifications
You must be signed in to change notification settings - Fork 131
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Mempool syncing optimizations & fixups #89
Commits on Sep 10, 2024
-
Don't use RPC batching with bitcoind
This actually hurts performance because the batched response has to be bueffered on the bitcoind side, as @TheBlueMatt explains at romanz#373 (comment) Instead, send multiple individual RPC requests in parallel using a thread pool, with a separate RPC TCP connection for each thread. Also see romanz#374
Configuration menu - View commit details
-
Copy full SHA for 5b07357 - Browse repository at this point
Copy the full SHA 5b07357View commit details -
Don't store mempool txs before all prevouts are available
The indexing process was adding transactions into the store so that prevouts funded & spent within the same batch could be looked up via Mempool::lookup_txos(). If the indexing process later failed for any reason, these transactions would remain in the store. With this change, we instead explicitly look for prevouts funded within the same batch, then look for the rest in the chain/mempool indexes and fail if any are missing, without keeping the transactions in the store.
Configuration menu - View commit details
-
Copy full SHA for 5c8c785 - Browse repository at this point
Copy the full SHA 5c8c785View commit details -
Continuously attempt to fetch mempool transactions
Previously, if any of the mempool transactions were not available because they were evicted between getting the mempool txids and txs themselves, the mempool syncing operation would be aborted and tried again from scratch. With this change, we instead keep whatever transactions we were able to fetch, then get the updated list of mempool txids and re-try fetching missing ones continuously until we're able to get a full snapshot.
Configuration menu - View commit details
-
Copy full SHA for 4197301 - Browse repository at this point
Copy the full SHA 4197301View commit details -
Configuration menu - View commit details
-
Copy full SHA for e3481c7 - Browse repository at this point
Copy the full SHA e3481c7View commit details
Commits on Sep 11, 2024
-
- Reduce logging level for bitcoind's JSONRPC response errors These can happen pretty often for missing mempool txs and do not warrant warn-level logging. Unexpected RPC errors will bubble up and be handled appropriately. - Add more verbose logging for mempool syncing
Configuration menu - View commit details
-
Copy full SHA for ebcd737 - Browse repository at this point
Copy the full SHA ebcd737View commit details -
Reuse RPC threads and TCP connections
Keep RPC TCP connections open between sync runs and reuse them, to reduce TCP connection initialization overhead.
Configuration menu - View commit details
-
Copy full SHA for aee32b6 - Browse repository at this point
Copy the full SHA aee32b6View commit details -
Avoid recomputing txids when possible
Following Blockstream#89 (comment) and Blockstream#89 (comment)
Configuration menu - View commit details
-
Copy full SHA for 7a068bf - Browse repository at this point
Copy the full SHA 7a068bfView commit details