# Period Store ## Period Store ### Features When using the prover as a remote prover, there are serveral features, which will be supported. #### 1. Historical Proofs When creating proofs for block older than the current sync\_committee you can't prove them with the current keys unless you use the historical\_summaries in the state of the Beacon chain. In order to do this, you will not only need a beacon API capable of providing proof for the historical summaries, which are currently only supported by Lodestar and nimbus, but also require all 8192 blockroots in order to build the merkle-proof. In order to be able to build those proofs on demand, the server manages a period\_store, where those block roots are stored in one file. (blocks.ssz) The server is also keeping track of new heads on continue to write those block roots so it can create those historical proofs when needed. This would hardly be possible on a local device. #### 2. ZK-Proofs for Sync-Committee Transitions. The verifier always needs the 512 pubkeys of the current sync\_committee in order to verify the signatures of the blocks. In order to get them, there are 2 ways: 1. setting or getting a current checkpoint from a checkpoint provider and the getting the proof from a beacon API for the sync\_committees keys (light client bootstrap). But of checkpoint would need to come from a external source you need to trust. And if your client was offline for a longer time, you would need to use the light\_client\_updates, where each would be 25kB per 27h, to prove the transitions to the current keys. 2. create a hardcoded fixed trust anchor and the use a zero-proof to prove all the transitions from that point to the current sync\_committee. This also allows to create such a prove offline and is independent of how old this trsut anchor is. Of course in both cases a weak subjectivity check may be needed, but even this can be done with signed checkpoints, so you would be able to create proofs without relying on external source and so support offline verification. ### Architecture A **period** is a contiguous range of **8192 beacon slots** (≈ 27h on mainnet). * Period number: `period = slot >> 13` (equivalent to `slot / 8192`). * The server maintains a local on-disk directory called the **period store**. * The period store enables: * **Historical proofs** (needs all block roots of a period to build Merkle proofs) * **Light client caching** (LCU/LCB per period) * **ZK proofs** for sync committee transitions (pre-generated by the master) ### Directory layout `period_store` is a directory with one subdirectory per period: ``` / / blocks.ssz headers.ssz blocks_root.bin historical_root.json lcu.ssz lcb.ssz sync.ssz zk_pub.bin zk_proof.bin zk_proof_g16.bin zk_vk.bin zk_vk_raw.bin zk_groth16.bin ``` #### Files (what they mean) * `blocks.ssz` * 8192 block roots, each 32 bytes → max \~256 KiB. * Written incrementally slot-by-slot. * Required to build historical Merkle proofs. * `headers.ssz` * 8192 beacon block headers, each 112 bytes → max \~896 KiB. * Written incrementally slot-by-slot. * Used for integrity checks and to access historical headers. * `blocks_root.bin` * Marker file: contains the Merkle root of `blocks.ssz`. * Created only after `blocks.ssz` was verified against `historical_summaries`. * Semantics: if present, `blocks.ssz`/`headers.ssz` for that period are considered finalized and should not change. * `historical_root.json` * Proof response containing the `historical_summaries` (used to verify `blocks.ssz`). * `lcu.ssz` * Light client update for the period. * `lcb.ssz` * Light client bootstrap for the first finalized block in that period. * ZK-related files (produced by `scripts/run_zk_proof.sh`): * `sync.ssz`: prepared input data (temporary, may exist only during proof generation) * `zk_pub.bin`: public input data (e.g. `[SRC_PUBKEY_HASH:u256 | DST_PUBKEY_HASH:u256 | DST_PERIOD:u64 | ATTESTED_HEADER_ROOT:u256 | DOMAIN:bytes32]`) * `zk_proof.bin`: SP1 core proof * `zk_proof_g16.bin`: Groth16 proof (used by the verifier) * `zk_vk.bin`, `zk_vk_raw.bin`, `zk_groth16.bin`: verification keys / proof artifacts * `zk_proof.ssz`: Packs `zk_proof_g16.bin` into the SSZ container used for sync-data delivery (`ZKSyncData`, see `C4_ETH_REQUEST_SYNCDATA_UNION[2]` in `src/chains/eth/ssz/verify_types.c`). This also includes the next checkpoint and all headers needed to prove the `attestedHeader`. ### Writer model #### Head tracking (continuous) On every new head event, the server writes: * the slot’s block root into `blocks.ssz` * the slot’s header into `headers.ssz` This happens for the current period and is safe against reorgs by overwriting the slot position. #### Backfill (startup + periodic) Backfill exists to quickly populate the last N periods after startup (or after falling behind). Configuration: * `period_backfill_max_periods` (`C4_PERIOD_BACKFILL_MAX_PERIODS`) * `period_backfill_delay_ms` (`C4_PERIOD_BACKFILL_DELAY_MS`) How it works (high level): * once the master-server receives the first head and knows the latest block root, it will trigger a backfill, checking whether all the root doen to max\_periods are stored and correct. Missing blocks will be fetched from the beacon API. * A delay is applied between backfill requests to avoid public API rate limits. #### Blocks root verification (`blocks_root.bin`) Once `historical_root.json` exists for a historic period, the server can verify: * hash-tree-root(`blocks.ssz`) == expected root from `historical_summaries` If the verification succeeds, it writes `blocks_root.bin` as an immutable marker. ## Master / Slave modes The mode is determined by configuration: * **Master mode**: `period_store` configured and `period_master_url` **not** set. * Writes period store data locally. * Serves period store over HTTP. (/period\_store) * Generates ZK proofs. * **Slave mode**: `period_store` configured and `period_master_url` **set**. * Does not generate ZK proofs. * Fetches data from the master. * Optionally performs `full_sync`. ### HTTP API: `/period_store` The server exposes the period store via `/period_store`. #### File download ``` GET /period_store// GET /period_store//?offset= ``` * `offset` is optional. * If provided, bytes are served starting at that offset (used for resumable downloads). #### Manifest (SSZ) ``` GET /period_store?manifest=1&start= ``` Returns `application/octet-stream` containing an SSZ list of: * `period: uint64` * `filename: SSZ_STRING(32)` (null-terminated) * `length: uint32` Notes: * The master **fails fast** if period directory gaps are detected. * The handler caches small manifest responses for \~30s to reduce repeated scanning from multiple slaves. * Individual files larger than a small safety limit (currently 5 MiB) are treated as an error for the manifest. ### Slave sync: lazy vs full sync #### Lazy fetch (default) When a slave needs a file that is missing locally, it performs an on-demand download from the master and stores it in its own `period_store`. This minimizes transfer volume but does not guarantee a full backup. **Full sync (optional)** If enabled, the slave tries to keep a **complete** local replica of the master’s period store. Configuration: * `C4_PERIOD_FULL_SYNC=1` / `period_full_sync=1` * `PERIOD_MASTER_URL` / `period_master_url=` Trigger: * The full sync is triggered on finalized checkpoints (≈ every 6 minutes on mainnet). Algorithm (high level): 1. Determine `last_full_period` by scanning local directories from newest to oldest until a “complete” period is found: * `blocks_root.bin` present * `zk_proof_g16.bin` present 2. Request manifest from master: ``` GET /period_store?manifest=1&start= ``` 3. For each manifest entry, download the file if needed: * If local size == manifest length: skip * If local size < manifest length: download with `?offset=` and append * If local size > manifest length: full download (truncate) Micro-fork edge case: * If `blocks_root.bin` exists for a period on the master, `blocks.ssz` and `headers.ssz` are always re-downloaded fully (truncate) to avoid missing silent corrections. Failure handling: * If a download/write fails for a period that already looks “complete”, the slave removes `blocks_root.bin` so the period will be revisited on the next sync. ### ZK proofs: generation and configuration ZK proofs are generated only on the **master**. #### Trigger On each finalized checkpoint, the master checks whether the next period’s proof exists and is valid. * Target period for proving: `target_period = current_period + 1` * Proof files: * `zk_proof_g16.bin` * `zk_pub.bin` If a proof exists: * It is verified. * If invalid: * If “fresh” (< 1h): do not re-prove (to avoid cost loops) * If “old”: delete and retry #### Configuration * `PERIOD_PROVER_KEY_FILE` / `period_prover_key_file` * Path to a file containing the SP1/Network private key (e.g. via Docker secrets). * The server passes this path to the proof script via `SP1_PRIVATE_KEY_FILE=`. #### Script execution The master spawns `scripts/run_zk_proof.sh` (or `/app/run_zk_proof.sh` in Docker) with arguments similar to: ```bash run_zk_proof.sh \ --period \ --prev-period \ --prove \ --groth16 \ --network \ --output ``` The script: * Loads keys from `SP1_PRIVATE_KEY_FILE` (or `/run/secrets/...` fallbacks) * Uses prebuilt artifacts in Docker if present (`/app/eth_sync_program`, `/app/eth-sync-script`) * Produces the proof artifacts inside `//...` ### Minimal configuration examples #### Master ```bash export DATA=/data/period_store unset PERIOD_MASTER_URL export PERIOD_PROVER_KEY_FILE=/run/secrets/sp1_private_key ``` #### Slave (lazy fetch only) ```bash export DATA=/data/period_store export PERIOD_MASTER_URL=https://master.example export C4_PERIOD_FULL_SYNC=0 ``` #### Slave (full sync) ```bash export DATA=/data/period_store export PERIOD_MASTER_URL=https://master.example export C4_PERIOD_FULL_SYNC=1 ``` --- # Agent Instructions: Querying This Documentation If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question. Perform an HTTP GET request on the current page URL with the `ask` query parameter: ``` GET https://corpus-core.gitbook.io/specification-colibri-stateless/specifications/ethereum/period-store.md?ask= ``` The question should be specific, self-contained, and written in natural language. The response will contain a direct answer to the question and relevant excerpts and sources from the documentation. Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.