Compact Block Filters (Neutrino)

What Are Compact Block Filters?

Compact block filters are a privacy-preserving technique that allows lightweight Bitcoin clients to determine which blocks contain transactions relevant to their wallet. Defined in BIP 157 (the peer-to-peer protocol) and BIP 158 (the filter data structure), compact block filters invert the traditional SPV filtering model: instead of sending address information to a full node, the client downloads a small, compressed summary of each block and checks it locally.

The protocol is often called "Neutrino" after Lightning Labs' reference implementation. It was designed by Olaoluwa Osuntokun (roasbeef), Alex Akselrod, and Jim Posen to solve a fundamental problem: earlier light client protocols like BIP 37 bloom filters leaked wallet addresses to serving nodes, undermining the privacy of mobile and resource-constrained users.

Each filter is a Golomb-coded set (GCS) that compactly encodes the scriptPubKeys present in a block. A typical filter is a few hundred bytes to around 15 KB: far smaller than the full block it summarizes. Clients download these filters, test their own addresses against them, and only fetch full blocks when a match occurs.

How It Works

The compact block filter system has two layers: BIP 158 defines how filters are constructed from block data, and BIP 157 defines how clients request and verify those filters over the peer-to-peer network.

Filter Construction (BIP 158)

A full node builds one filter per block using the following process:

1. Collect every scriptPubKey from the block's outputs (excluding OP_RETURN) and every scriptPubKey being spent by the block's inputs (excluding the coinbase transaction)
2. Hash each script to a 64-bit integer using SipHash, keyed by the first 16 bytes of the block hash
3. Map each hash into the range [0, N × M), where N is the number of items and M is 784,931
4. Sort the mapped values and compute the differences between consecutive entries
5. Encode the differences using Golomb-Rice coding with parameter P = 19, producing a compact bitstream

The result is a deterministic, compressed set that any node will produce identically for the same block. The false positive rate is approximately 1 in 784,931 per element tested: a client checking an address against a filter has a very small chance of a spurious match, which simply means downloading one unnecessary full block.

Golomb-Coded Sets

Golomb-coded sets are a probabilistic data structure similar to bloom filters, but with a key difference: they are immutable and produce a fixed-size output for a given set. The encoding works by sorting hashed values and storing only the gaps between them. Since these gaps follow a geometric distribution, Golomb-Rice coding compresses them efficiently. The structure is related to the Merkle trees used in Bitcoin block headers: both are compact representations that allow verification without holding all underlying data.

Each gap is divided by 2^P (where P = 19) to produce a quotient and remainder. The quotient is encoded in unary (a run of ones followed by a zero), and the remainder is stored as P bits. This produces compact filters that are roughly 1/100th the size of the blocks they summarize.

# Golomb-Rice encoding of a gap value
gap = 650000
P = 19

quotient = gap >> P        # gap / 2^19 = 1
remainder = gap & 0x7FFFF  # gap mod 2^19 = 125568

# Encode: quotient in unary (10), remainder in 19 bits
# Result: 10 + 0011110101010000000

Client-Side Matching

A light client syncs with the network in several phases:

1. Download and validate the full chain of block headers (approximately 40 MB for the entire Bitcoin history)
2. Download filter headers: each is the double-SHA256 of the filter hash concatenated with the previous filter header, forming a verifiable chain
3. Connect to multiple peers and compare filter headers to detect dishonest nodes: if peers disagree, download the full block and recompute the filter locally to identify the liar
4. Download the actual compact filters and test wallet scriptPubKeys against each one using the same SipHash parameters
5. When a filter matches, fetch the full block from a different random peer (to reduce privacy leakage) and extract relevant transactions

P2P Protocol Messages (BIP 157)

BIP 157 introduces six new messages for requesting and serving filters:

Full nodes signal support by setting the NODE_COMPACT_FILTERS service bit. Only one filter type is currently defined: the "basic" filter (type 0x00). An extended filter type was originally proposed but removed before finalization due to lack of clear use cases.

Compact Block Filters vs. Bloom Filters

BIP 37 bloom filters, used by early SPV wallets like those built on BitcoinJ, follow a fundamentally different model: the client constructs a bloom filter containing its addresses and sends it to a full node, which filters transactions server-side. Research has demonstrated that this approach offers "virtually zero privacy," because typical implementations include both the public key and its hash in the filter, dramatically reducing the effective false positive rate and letting the server deduce wallet addresses.

Compact block filters eliminate these problems by shifting all filtering to the client. The server produces the same deterministic filter for every requester, gaining no information about which addresses any particular client controls. The tradeoff is bandwidth: compact block filter clients download every filter regardless of relevance, whereas bloom filter clients only receive matching transactions.

Why It Matters

Compact block filters solve a practical problem for anyone building a self-custodial Bitcoin wallet on a mobile device. Running a full Bitcoin node is impractical on a phone: the blockchain exceeds 600 GB. But relying on a third-party server to watch your addresses means trusting that server with your privacy and trusting it to report honestly. For a broader comparison of these tradeoffs, see the self-custodial vs. custodial wallets research article.

With compact block filters, a mobile wallet can verify blockchain state independently while downloading only a few hundred megabytes of data. The client never reveals which addresses belong to the user, and it can detect if a server is lying by cross-checking filters from multiple peers. This makes genuine self-custody viable on resource-constrained devices.

For layer-2 protocols like Lightning and Spark, compact block filters are especially important. Lightning nodes must monitor the blockchain for critical on-chain events: channel opens, force-closes, and HTLC timeouts. Missing a force-close transaction could result in lost funds. Unlike bloom filters, where a malicious server could silently omit a critical transaction, compact block filters let the client verify filter completeness by comparing responses from multiple peers.

Implementations

Neutrino (Lightning Labs)

Neutrino is the reference client-side implementation, written in Go and maintained by Lightning Labs. It was built specifically for LND and is designed to run on mobile devices. When LND operates in "Neutrino mode," it connects to compact-block-filter-serving peers instead of requiring a local full node.

# Run LND with Neutrino backend
lnd --bitcoin.active --bitcoin.mainnet \
    --bitcoin.node=neutrino \
    --neutrino.connect=btcd-node.example.com

Bitcoin Core

Bitcoin Core added server-side compact block filter support incrementally:

• v0.19 (2019): BIP 158 filter construction and the blockfilterindex
• v0.21 (2021): serving filters over P2P via the -peerblockfilters flag
• v25.0: fast wallet rescans using filters and the scanblocks RPC command

# Enable compact block filter serving in bitcoin.conf
blockfilterindex=1
peerblockfilters=1

Building the filter index requires approximately 5 GB of additional disk space and about one hour of initial computation.

Kyoto (BDK)

Kyoto is a Rust implementation of BIP 157/158 designed for the Bitcoin Dev Kit (BDK) ecosystem. It targets mobile devices with a low memory footprint (30 to 80 MB during sync). The bdk-ffi 1.2 release brought compact block filter support to iOS and Android developers through foreign language bindings, making privacy-preserving light clients accessible to the mobile HD wallet ecosystem.

Use Cases

Lightning Mobile Wallets

Non-custodial Lightning wallets are the primary use case for compact block filters. Wallets like Blixt and Breez run full LND nodes on mobile devices, using Neutrino to sync blockchain state. This architecture lets users send and receive Lightning payments without trusting a custodian or leaking address information to a server. For a deeper look at these designs, see the Lightning mobile wallets architecture research article.

Private Wallet Recovery

When recovering an HD wallet from a seed phrase, the wallet must scan the blockchain for all addresses derived from its derivation paths. Using compact block filters, the wallet can perform this scan without revealing which addresses it controls to any server. A full recovery scan downloads approximately 1 GB of filter data and completes in minutes on a modern connection.

Lightweight On-Chain Monitoring

Applications that need to watch for specific on-chain events (confirmations, spends, new outputs at known scripts) can use compact block filters to monitor efficiently. This is useful for watchtowers that guard Lightning channels, channel reserve monitoring, and any service that tracks UTXOs without running a full archival node. For more on how watchtowers protect Lightning users, see the watchtower deep dive.

Risks and Considerations

Bandwidth Overhead

Compact block filter clients download every filter for every block, regardless of whether the block is relevant. This is the cost of privacy: the server cannot know which filters the client actually needs, so it sends them all. Ongoing monthly sync requires approximately 70 MB of filter data. For users on metered connections, this overhead may be significant compared to Electrum-style servers that send only matching transactions.

Limited Serving Nodes

Because blockfilterindex and peerblockfilters are not enabled by default in Bitcoin Core, only a fraction of the network (roughly 1,000 out of approximately 11,000 reachable nodes) currently serve compact block filters. This can impact connection reliability for light clients, particularly during initial sync. Running a full node with these options enabled contributes to the network's light client infrastructure.

No Filter Commitment in Blocks

Unlike block headers, compact block filters are not committed to in the Bitcoin block structure. A client cannot cryptographically prove that a filter is correct without downloading the full block and recomputing it. The security model relies on connecting to multiple honest peers: if all connected peers collude, they could serve false filters. Adding a filter commitment (for example, in the coinbase transaction) would strengthen security but would require a soft fork.

Block Download Privacy Leak

While filter downloading reveals nothing about the client's wallet, fetching a full block after a filter match does signal to the serving peer that the block likely contains a relevant transaction. Mitigations include downloading matched blocks from random outbound peers and routing requests through anonymity networks, but this residual leak is inherent to the protocol's design. For more on Lightning-specific privacy tradeoffs, see the Lightning Network privacy analysis.

This glossary entry is for informational purposes only and does not constitute financial or investment advice. Always do your own research before using any protocol or technology.