Consensus Neotrace

Problem Statement

The Problem: The Ethereum Consensus Specification (written in Python) is the source of truth for the Beacon Chain. However, client teams (Prysm, Lighthouse, Teku, etc.) write their implementations in Go, Rust, and Java. Ensuring these clients perfectly match the Python reference requires thousands of test vectors. Currently, creating complex state transition vectors is often manual, tedious, or limited to specific scenarios.The Solution: We built a "Smart Execution Recorder" that hooks directly into the Python executable spec. By simply running the existing Python unit tests with our @record_spec_trace decorator, the system automatically generates standardized, cross-client trace.yaml test vectors.It doesn't just dump data; it intelligently tracks the Beacon State. It detects when the state is mutated (even in-place), calculates Merkle root hashes to deduplicate state snapshots, and produces a content-addressed, portable trace file that any client can use to verify their execution against the spec, step-by-step.

Solution

The project is built using Python and integrates deeply with the ethereum/consensus-specs repository.Transparent Proxies (wrapt): We use object proxies to wrap the spec module. This allows us to intercept every function call, argument, and return value without modifying the core specification logic.Smart State Tracking: A core challenge was tracking the 32GB+ Beacon State efficiently. We implemented a "smart tracker" that calculates the hash_tree_root of the state before and after every operation. It detects context switches and out-of-band mutations, only serializing the state when the content actually changes.Content-Addressing: All SSZ objects (States, Blocks, Attestations) in the trace are identified by their raw hex Merkle root. This ensures valid deduplication and allows the trace to be "replayable" by clients who can verify they arrived at the exact same state root.Pydantic Schema: We use Pydantic models to define the trace.yaml schema. We implemented custom validators to automatically sanitize Python-specific types (like Slot or ValidatorIndex subclasses) and raw bytes into standard, portable hex strings that other languages can parse easily.Pytest Integration: The system hooks into the existing test runner via a custom decorator, making it zero-config to generate a trace for any existing test case.