BIP 157/158 (Client-Side Block Filtering)

What Is BIP 157/158?

BIP 157 and BIP 158 are two complementary Bitcoin Improvement Proposals that together enable client-side block filtering: a technique that lets light clients determine whether a block contains relevant transactions without downloading the entire block and without revealing their interests to the serving node.

BIP 158, authored by Olaoluwa Osuntokun and Alex Akselrod, specifies how to construct compact probabilistic filters for each block using Golomb-Rice coding. BIP 157, authored by the same pair along with Jim Posen, specifies the peer-to-peer network messages that light clients use to request these filters from full nodes.

The core insight is a reversal of who holds the filter. Under BIP 37, the client sends its filter to the server, effectively telling the server what it is looking for. Under BIP 157/158, the server sends a deterministic filter to the client, and the client checks it locally. The server learns nothing about the client's wallet.

Why BIP 37 Bloom Filters Failed

BIP 37 was Bitcoin's original protocol for SPV light clients. The client would construct a Bloom filter containing its addresses and send it to a full node. The node would then stream only the transactions matching the filter, saving the client from downloading entire blocks.

The privacy model relied on the Bloom filter's false-positive rate: if the filter matched some transactions the client did not care about, the server could not tell which matches were real. In practice, this protection collapsed for several reasons:

• Wallets often set low false-positive rates to save bandwidth, making the filter precise enough for the server to identify the client's addresses
• When clients reconnected and sent updated filters, servers could intersect the old and new filters to eliminate false positives
• Servers could combine filter data with blockchain transaction-graph analysis to deanonymize wallets
• The protocol was also vulnerable to denial-of-service attacks: a malicious client could force a server to scan every block for matches, consuming significant CPU

Bitcoin Core disabled BIP 37 Bloom filter serving by default in version 0.19.0, effectively deprecating the protocol. BIP 157/158 was designed as the replacement.

How It Works

The system has two layers: BIP 158 handles filter construction, and BIP 157 handles filter delivery over the network.

BIP 158: Building the Filter

For each block, a full node builds a compact filter over the set of scripts that appear in that block. The basic filter type (type 0x00) includes all scriptPubKeys from the outputs created by the block's transactions, plus all scriptPubKeys spent by the block's inputs. This captures every address that sends or receives funds in that block.

The construction process works as follows:

1. Collect all relevant script data items from the block
2. Hash each item using SipHash (with parameters c=2, d=4) keyed with the first 16 bytes of the block hash
3. Map each 64-bit hash output uniformly into the range [0, N × M), where N is the number of items and M is the false-positive parameter
4. Sort the mapped values in ascending order
5. Compute the differences (deltas) between consecutive sorted values
6. Encode each delta using Golomb-Rice coding with parameter P, which splits each value into a quotient (encoded in unary) and a remainder (encoded in P bits)

For the basic filter type, the spec sets P = 19 and M = 784,931. These values were chosen to minimize total bandwidth: too low an M means the client downloads too many false-positive blocks, while too high an M makes the filters themselves large. The optimal M is approximately 1.497137 × 2^P, which yields 784,931 when P = 19. The resulting false-positive rate is 1/M, or roughly 1 in 784,931 per element queried.

BIP 157: Fetching Filters Over the Network

BIP 157 adds six new P2P network messages for light clients to request and receive filters:

• getcfilters / cfilter: request and receive compact filters for a range of blocks (up to 1,000 per request)
• getcfheaders / cfheaders: request and receive filter headers, which chain together via hashing to form a commitment that clients use to verify filter integrity
• getcfcheckpt / cfcheckpt: request and receive evenly spaced filter-header checkpoints for fast verification of large header chains

A full node advertises its willingness to serve filters through the NODE_COMPACT_FILTERS service bit. When a light client connects, the typical sync flow is:

1. Download block headers via the standard headers-first sync protocol
2. Fetch filter-header checkpoints via getcfcheckpt from multiple peers to detect dishonest nodes
3. Fetch the full chain of filter headers via getcfheaders and verify they chain correctly to the checkpoints
4. Fetch the actual compact filters via getcfilters for the range of blocks the client needs
5. For each filter, check whether any of the client's addresses or scripts match. If a filter matches, download the full block from a peer and extract the relevant transactions

The filter headers form a hash chain: each header is the double-SHA-256 of the filter's hash concatenated with the previous filter header. This lets clients detect tampered or omitted filters by cross-checking headers from multiple peers.

Technical Example

A light client checking whether a block contains transactions for its addresses:

# Client holds a set of watched scripts (addresses)
watched_scripts = [script_a, script_b, script_c]

# For each new block, fetch its compact filter
filter = fetch_cfilter(block_hash, filter_type=0x00)

# Derive the SipHash key from the block hash
key = block_hash[:16]  # first 16 bytes, little-endian

# Test each watched script against the filter
if filter.match_any(key, watched_scripts):
    # Possible match (may be false positive)
    # Download the full block to confirm
    block = fetch_full_block(block_hash)
    for tx in block.transactions:
        check_for_relevant_outputs(tx, watched_scripts)
else:
    # No match: skip this block entirely
    pass

Neutrino: The Reference Implementation

Neutrino is the reference implementation of BIP 157/158, built by Lightning Labs in Go. It functions as a Bitcoin light client library that syncs the blockchain using compact block filters rather than downloading full blocks or relying on trusted third-party APIs.

Lightning Labs integrated Neutrino into LND starting with version 0.5-beta in September 2018. In Neutrino mode, an LND node syncs the Bitcoin blockchain by downloading only block headers and compact filters, then fetching full blocks only when a filter matches a channel-relevant transaction. This dramatically reduces storage and bandwidth requirements while maintaining privacy.

For mobile Lightning wallets and applications built on LND, Neutrino eliminates the need to trust a specific server with wallet address data. A wallet using Neutrino connects directly to the Bitcoin P2P network, downloads filters from multiple peers, and verifies everything locally.

Use Cases

Mobile and Resource-Constrained Wallets

Mobile Bitcoin wallets cannot run full nodes due to storage, bandwidth, and battery constraints. Before BIP 157/158, these wallets had two options: trust a centralized server (Electrum-style) or use BIP 37 Bloom filters (which leak address data). Compact block filters provide a third path: light clients that are both private and trustless.

Lightning Network Light Clients

Lightning nodes need to monitor the blockchain for channel-related transactions: funding transactions, cooperative closes, force closes, and potential breach attempts. A lightweight client using BIP 157/158 can watch for these events without a full node, enabling Lightning on mobile devices. This is how LND's Neutrino mode operates, and it has been a key enabler for mobile Lightning adoption.

Privacy-Conscious Blockchain Scanning

Even users who run a full node at home may want a private light client on their phone. BIP 157/158 ensures that the phone does not leak wallet activity to whichever peers it connects to. The user gets convenience without sacrificing privacy.

Tradeoffs and Considerations

Bandwidth Cost

With BIP 37, the server only sent matching transactions. With BIP 157/158, the client downloads a filter for every block, regardless of whether the block is relevant. Each basic filter is roughly 15 to 20 KB for a typical block, and syncing the full chain from genesis requires downloading the entire filter chain.

This is an intentional tradeoff: the client pays more bandwidth in exchange for revealing nothing to the server. The total filter chain is still far smaller than the full blockchain. For a client that only needs recent blocks (such as a Lightning wallet), the overhead is modest.

False Positives Still Require Full Block Downloads

When a filter indicates a match, the client must download the full block to confirm. With a false-positive rate of about 1 in 784,931 per element, a wallet watching a small number of scripts will rarely encounter false positives. However, a wallet with thousands of addresses increases its chance of false matches, leading to more unnecessary block downloads.

No Filter for Transaction Completeness

A malicious full node could serve a valid but incomplete filter (omitting a script that should be included). The filter-header chain lets the client detect disagreements between peers: if two peers serve different filter headers for the same block, the client knows one is dishonest and can download the full block to verify. This requires connecting to multiple honest peers, which is a weaker security assumption than running a full node.

Server-Side Resource Costs

Full nodes that serve compact block filters must build and store an additional index. Unlike BIP 37, where the server scans blocks on demand, BIP 158 filters are precomputed and deterministic: every honest node produces the same filter for the same block. This makes serving efficient but adds storage requirements for the filter and filter-header indexes.

BIP 157/158 vs. Other Light Client Approaches

Adoption

Both BIPs carry "Deployed" status in the BIP repository. Bitcoin Core added support for building and serving compact block filters, enabled via the -peerblockfilters configuration flag in conjunction with -blockfilterindex.

LND's Neutrino mode remains the most prominent consumer of BIP 157/158 filters. Other Bitcoin libraries and light-client implementations, including LDK and BDK, have explored integration with varying degrees of completeness. The protocol's privacy guarantees make it particularly relevant for light-client architectures in mobile wallets and second-layer systems like Lightning and Spark, where monitoring the base chain without leaking user data is essential.

This glossary entry is for informational purposes only and does not constitute financial or investment advice. Always do your own research before using any protocol or technology.