A hidden Bitcoin node database just shrank by 40 GB, but only if runners rebuild it

Bitcoin Core has merged a redesign of its optional transaction index that cut about 40 GB from the database in one contributor’s mainnet test. Operators using -txindex retain their existing indexes through an upgrade; capturing the full saving requires recreating the database.

Pull request #35531, merged into Bitcoin Core’s master branch on Aug. 15, reduced the author’s rebuilt mainnet txindex from about 66 GB to 26 GB. The roughly 61% reduction is confined to this optional index. Bitcoin’s blockchain and the rest of a node’s data directory remain outside the measurement.

The code is merged upstream. Stable binaries follow a separate release process, and Bitcoin Core’s release index leaves the first version containing the change unspecified. Operators will need the migration notes for the release that ships it.

How the smaller index works

Bitcoin Core’s -txindex option maintains a database for retrieving transactions by their full transaction ID. The old format stored each 32-byte transaction ID as a database key alongside transaction disk-position data.

The redesign stores a much shorter lookup key: a five-byte prefix derived from a salted SipHash, followed by a six-byte suffix encoding the block sequence and transaction offset. The full transaction ID is still checked before Bitcoin Core returns a match.

That verification step protects against collisions created by the shorter prefix. Bitcoin Core scans entries sharing the prefix, locates candidate blocks through its block index, reads the candidate transactions from disk and compares their full IDs. Bitcoin Optech’s technical summary described collisions as extra read and verification work, with full-ID checks preventing false transaction matches.

Performance held steady in the author’s test. Lookups took about 0.2 milliseconds. The mainnet rebuild finished in 1 hour 19 minutes versus 1 hour 50 minutes with the prior format. Hardware, storage, chain height and software version can all change those results.

Existing txindex databases remain readable after an upgrade, avoiding an immediate forced rebuild. Their legacy entries also keep the larger footprint, so realizing the full 40 GB benchmark saving requires recreating the index.

A later downgrade carries a second migration cost. Bitcoin Core’s merged release-note fragment says previous releases cannot read entries written in the compact format. Returning to an older release after rebuilding would trigger another txindex rebuild in the old format.

The operator payoff is substantial within its narrow scope: a much smaller optional index and a faster rebuild in the contributor’s test. Capturing it requires planned recreation of the database, plus another rebuild if a rollback becomes necessary.

Release-specific notes should control the exact recreation and downgrade procedure once the change reaches a stable Bitcoin Core binary.

The post A hidden Bitcoin node database just shrank by 40 GB, but only if runners rebuild it appeared first on CryptoSlate.