For the complete documentation index, see llms.txt. This page is also available as Markdown.

Python

Python bindings for the Colibri stateless Ethereum proof library. Generate and verify cryptographic proofs for Ethereum RPC calls without trusting centralized infrastructure.

Overview

The Colibri Python Bindings provide a modern, async-first Python API for verified blockchain interactions. Built with pybind11 for optimal performance and memory management, these bindings enable secure Web3 functionality without dependency on centralized RPC providers.

Core Features

  • 🔐 Cryptographic Verification - All RPC responses validated with Merkle proofs

  • 🚀 Async/Await Support - Modern Python async support for network operations

  • 💾 Pluggable Storage - Customizable storage backends for caching

  • 🧪 Comprehensive Testing - Mock HTTP requests and storage for testing

  • 🌐 Multi-Chain Support - Ethereum Mainnet, Sepolia, Gnosis Chain, and more

  • 📦 Easy Installation - Simple pip install with pre-built native extensions

Architecture

┌─────────────────────────────────────────────────────────────────┐
│                    Python Application Layer                     │
├─────────────────────────────────────────────────────────────────┤
│                     colibri.client API                          │
│  • Colibri class (main interface)                               │
│  • Async RPC methods                                            │
│  • Storage & HTTP abstractions                                  │
│  • Error handling & type conversion                             │
├─────────────────────────────────────────────────────────────────┤
│                  Python-C++ Bridge Layer                        │
│  • _native.so (pybind11 extension)                              │
│  • Function pointer callbacks                                   │
│  • Memory management & cleanup                                  │
├─────────────────────────────────────────────────────────────────┤
│                   Core C Libraries                              │
│  • Prover (proof generation)                                   │
│  • Verifier (proof verification)                                │
│  • Storage plugin system                                        │
│  • Cryptographic libraries (blst, ed25519)                      │
└─────────────────────────────────────────────────────────────────┘

Installation

Pre-built wheels are available for:

  • Linux: x86_64

  • macOS: ARM64 (Apple Silicon) and x86_64 (Intel)

  • Windows: x86_64

Development Installation

Quick Start

Basic RPC Calls

Local Proof Generation

Multi-Chain Setup

API Reference

Colibri Class

Core Methods

Storage System

Built-in Storage Implementations

Default File Storage

Memory Storage

Custom Storage Implementation

Testing Framework

Mock Testing

Integration Testing

Custom Test Data

Configuration

Chain Configuration

Advanced Configuration

Prover Mode

Controls how proofs are built and verified. Set via prover_mode in the constructor:

  • ProverMode.LOCAL -- Proofs are built entirely on the client. Requires access to a Beacon API and execution layer RPC. Fully trustless, but slower and needs more infrastructure.

  • ProverMode.REMOTE -- Proofs are fetched from a remote Colibri prover server. Fastest option but relies on the prover server for proof generation. The verifier still cryptographically checks every proof.

  • ProverMode.HYBRID -- The consensus-layer proof (BlockHeaderProof) comes from the Colibri server, while execution-layer data (account proofs, storage, etc.) is fetched directly from the RPC provider. Best balance of performance and scalability -- the Colibri server only serves lightweight, cacheable header proofs while the heavy RPC load goes to your existing provider.

  • ProverMode.PROXY -- Like remote, but the client sends its own RPC and Beacon API URLs to the prover server. The server uses these endpoints instead of its own. Useful when the client has access to private or premium RPC providers.

  • ProverMode.LIGHT_CLIENT -- Like hybrid, with additional background polling of block headers to keep the cache warm. Call await start_light_client() / stop_light_client() to control polling (default interval: 12s). By default only the compact eth_getBlockHeader is fetched; pass full_block=True to fetch the full block (useful when many eth_getTransactionByHash / eth_getTransactionReceipt calls follow).

Default: ProverMode.REMOTE when prover URLs are configured, ProverMode.LOCAL otherwise.

Weak Subjectivity Period check

Whenever a sync crosses the Weak Subjectivity Period (WSP) -- typically ~2 to 4 months on Ethereum mainnet -- the verifier anchors the highest finalized header against an external checkpointz / Beacon API endpoint. The check applies to all three sync paths: verifier-driven Light Client updates, prover-supplied LCSyncData, and prover-supplied ZKSyncData. For ZKSyncData the verifier prefers configured witness signatures (checkpoint_witness_keys + matching signatures from the prover) and only falls back to checkpointz when no witness anchor is available.

  • skip_wsp_check (bool, default False) -- sets VERIFY_FLAG_SKIP_WSP_CHECK (bit 1 << 7) and disables the round-trip. SECURITY: only safe when another trust anchor (witness signatures, hard-coded checkpoint, signed package) is in place; raises the risk of long-range attacks across periods older than the WSP. See the threat model -- long range attacks for background.

Freshness window for latest proofs

Proofs that target the latest block tag remain cryptographically valid forever -- without a freshness window, a months-old proof could still be replayed as "current". The Python binding therefore reads time.time() and forwards now - max_latest_age_seconds to the verifier, which rejects proofs whose block timestamp is older with "proof for latest too old".

The gate covers the following RPC methods:

  • EVM: eth_call, eth_estimateGas, colibri_simulateTransaction

  • Account: eth_getBalance, eth_getCode, eth_getStorageAt, eth_getTransactionCount, eth_getProof

  • Block / header: eth_getBlockByNumber, eth_getBlockHeader, eth_blobBaseFee, eth_maxPriorityFeePerGas

  • Implicit-latest: eth_blockNumber

Account methods rely on a slim timestamp leaf inside the state proof which is only emitted by prover version ≥ 1.1.27; against older provers the verifier fails closed ("cannot verify freshness of latest block without block context").

  • max_latest_age_seconds (int, default 60 ≈ 5 Ethereum slots) -- upper bound on the accepted age. Set to 0 to disable the check (e.g. when using legacy proof formats that do not embed a block context).

Caveat: the gate fires only on "latest" (not "safe"/"finalized"). If the host wallclock is behind max_latest_age_seconds (containers without time sync, sandboxes with time = 0), the lower bound clamps to 0 and the check is silently disabled. Make sure your runtime has a synced clock or set max_latest_age_seconds=0 explicitly to acknowledge this state.

PAP mode: the freshness check also applies to PAP, where the call proof arrives via colibri_proofCall (same proof structure as a direct eth_call). This requires a prover that embeds the block context (≥ 1.1.15); against an older PAP proof without a block timestamp the check fails closed ("cannot verify freshness of latest block without block context"). Set max_latest_age_seconds=0 to opt out.

eth_getLogs: intentionally not covered. The proof witnesses individual log entries; the request range itself (up to latest) is not part of the proof yet. Tracked under issue #128 (full log-range proofs).

Privacy (PAP)

PAP (Pragmatic Adaptive Privacy) reduces intent leakage towards RPC/prover by using cached data when available and verifying afterwards.

  • privacy_modePrivacyMode.NONE (default) or PrivacyMode.BASIC. With BASIC, the verifier sets the PAP flag so that method-type and verification can use cached storage for optimistic execution (e.g. for eth_call); method type may depend on params.

This feature is still experimental!

Privacy-preserving eth_call (oblivious + PAP + hybrid)

For an eth_call where storage privacy is preserved end-to-end, use all three options below. Defaults keep oblivious_nodes empty (disabled).

Setting
Role

prover_mode=ProverMode.HYBRID

Only the block proof comes from the prover; storage/account data is fetched from RPC or oblivious node and verified locally. Remote mode would pull the full call proof from the server.

privacy_mode=PrivacyMode.BASIC

PAP: no eth_createAccessList on the prover (that would leak which slots you read). Storage is resolved optimistically in the local EVM; only eth_getProof requests go out.

oblivious_nodes

TEE RPC for those eth_getProof calls. Sets VERIFY_FLAG_OBLIVIOUS and PAP automatically when non-empty. See Oblivious Labs for how oblivious nodes use TEE and Oblivious RAM (ORAM).

Error Handling

Exception Types

Verified EVM reverts (RevertError)

When an eth_call (or similar EVM execution) is verified successfully but the EVM itself executed a REVERT, the binding raises RevertError. This is a fully verified outcome -- not a transport or proof failure -- and matches the Geth-style RPC error {"code": 3, "message": "execution reverted", "data": "0x..."}.

RevertError is a subclass of RPCError (with code = 3) and exposes the raw revert return data as a 0x-prefixed hex string in data. Callers typically ABI-decode this against the contract's error definitions (custom errors, Error(string), etc.). This is the mechanism that lets dApp libraries decode OffchainLookup (EIP-3668 / CCIP-Read) for example for the ENS off-chain resolver.

Graceful Degradation

Building from Source

Prerequisites

Build Process

Development Build

Running Tests

Performance Optimization

Connection Pooling

Storage Caching

Troubleshooting

Common Issues

Import Error: "No module named '_native'"

"Segmentation fault on exit"

This was a known issue with Python/C++ object lifetime. Fixed in current version.

RPC Calls Fail with Proof Errors

Windows Build Issues

Debug Mode

Memory Usage Monitoring

Platform Specifics

Linux Considerations

  • glibc version: Pre-built wheels require glibc 2.28+ (Ubuntu 20.04+)

  • Security: Runs in user space, no special permissions required

  • Performance: Native performance with direct C++ integration

macOS Considerations

  • Apple Silicon: Native ARM64 support with optimal performance

  • Intel Macs: x86_64 compatibility maintained

  • Code Signing: All native libraries are properly signed

  • Minimum Version: macOS 10.15+ (Catalina)

Windows Considerations

  • Unicode: Full UTF-8 support for all text operations

  • Path Length: Handles long file paths correctly

  • Permissions: No administrator privileges required

  • Minimum Version: Windows 10 (1809+)

CI/CD Integration

GitHub Actions

Docker Integration

Further Information

Last updated