Secret Shared Validator (SSV) — Phase 1 Testing Summary

Secret Shared Validator (SSV) — Phase 1 Testing Summary

Secret Shared Validator (‘SSV’) is a unique technology that enables the distributed control and operation of an Ethereum validator. SSV uses an MPC threshold scheme with a consensus layer on top that governs the network. Its core strength is in its robustness and fault tolerance which leads the way for an open network of staking operators to run validators in a decentralized and trustless way.

Phase 1 Testing

The first public milestone for SSV is a testnet of independent operators running a dedicated SSV network for Pyrmont validators. The high-level goal of the testnet was to split a validator key into ‘shares,’ distribute them between 12 testing partners, and jointly run a number of Pyrmont validators. This presented as a major milestone as the code base had to be mature enough to be stable and correct, with the obvious tradeoff of testing as early as possible.

Mid-April was selected for the start date. The testnet would run for 10 days. The goal was to run 2 validators — we ended up running 3 (with a 4th added shortly thereafter).

Side comment: We decided to make phase 1 testing as low key as possible to remove any potential anxiety from an “official” public testnet. The project and code weren’t tested before so it was better to have a quiet testnet to make sure everything was working before a full-blown public testnet.

Testing Partners

The testers needed to be technically adept in order to help diagnose issues and potential fixes. During the community call, a shortlist of potential testers was made. Keeping in mind the right balance between staking services, independent developers, and highly technical community members.
The following 12 groups and individuals are those that took part;

Collin (Consensys)
Dankrad (EF)
Carl (EF)
Aditya (EF)
StakeWise
Prysmatic labs
Sigma Prime (Lighthouse)
Phile.ETH
Chris.ETH
Oisin.ETH
ethStaker (Superphiz)

We couldn’t be happier with the mix of testers, everyone’s combined knowledge and efforts ended up being extremely valuable in finding potential setup issues.

Initial Considerations

Following the testing scope above we decided to make life easier by taking a few shortcuts:

1. Each operator received a .env file with all the configurations (and secret shares) already pre-generated (can be found here). Of course in a real-world setup this would not exist, some other medium of key exchange will need to be developed.
2. All SSV nodes were hardcoded to a single Pyrmont beacon node. This wouldn’t fly in a real-world scenario but it seemed too complicated to ask early testers to spin up their own Pyrmont nodes.
3. Aggregation and proposals were not supported. This was decided mostly to save development time as it wasn’t strictly necessary to test the core components of an SSV node (iBFT, key sharing, signature reconstruction, etc.).
4. Between iBFT instances, consistency wasn’t supported. This meant an SSV node would not wait for a previous instance (an epoch with duties) to be decided to move to a new instance (new epoch with duties). This is not safe!. We decided to let this slide for testing as it would have required heavy development around storage, syncing and more.

With that, Pyrmont validators #100157, #100158, and #100159 were deposited in advance to prepare for the testnet genesis epoch (#34500).

Phase 1 Testing Overview

Duration: 10 days
Partners: 12
Pyrmont SSV validators: 3 (#100157, #100158, and #100159)
Scope: setup, maintenance, attestation rate, and effectiveness.

KPIs:

• attestation rate > 90%
• attestation effectiveness > 90%

SSV operators per validator:

Raw data of the results can be found here, and a detailed summary below.

Setup and Maintenance

Testers were given documentation; to ease the setup of the SSV node, some customization suggestions were not supported (custom ports, DNS, etc.) but will be important for future development.

3 issues opened during testing are now resolved, requiring a node update.

• Issue #32: node crash when connection drops.
• Issue #35: period peer status logs.
• Issue #49: invalid signature. This was the most significant issue. It manifested in a log saying an invalid signature was received on the signature reconstruction phase (just after iBFT consensus). It appeared to be shown on every testers console log, although it seemed to not hurt attestation rate or effectiveness. The root cause was a message container object that indexed messages wrong, causing message processing by a wrong code component. A fix can be found here.

An important suggestion/comment from @CarlBeek was made regarding the “black box” feel the node currently has. This was one of the tradeoffs we made to move testing along faster; most of the setup was pre-made which reduced the feeling of control for testers. For the next testing phase, a basic explorer will be shipped to allow more control and transparency into the network’s inner workings.

Inclusion (Attestation) Rate

Inclusion rate is the measurement of attested, missed, and orphaned attestations.

• Attested is when an attestation is included in a block
• Missed is when an attestation isn’t included in a block (for example the validator is offline)
• Orphaned is when an attestation is included in a block which wasn’t selected / finalized into the canonical chain.

The goal is to maximize attestation rate as attestation duties carry rewards. The table below was calculated from the raw data and shows the SSV validators running at >99% attestation rate which is already tied with the best staking services out there.

Although these results are great, with an SSV setup one should expect a 100% attestation rate or close to it for long periods of time. We are still investigating the missed attestations and their cause.

Inclusion Distance (Attestation Effectiveness)

Inclusion distance directly affects how efficient an Ethereum validator is in generating rewards, mostly based on the maximum potential reward. It is calculated by how fast the validator’s attestation aggregated and was put into a block. The faster it is, the higher the reward.

In the chart above we’ve used a pivot table to count the number of times each inclusion distance (from 0–5, we’ve removed the rest for scale purposes) appears in the data.

Around 95% of the time the testing validators managed to include their attestation at distance 0 (max reward), 2.5% of the time at distance 1. This is higher than expected although slightly lower than the best staking services out there. We are still going through the potential causes but one that comes to mind is slow answers from the beacon node which can cause delays in the consensus and ultimately inclusion distance.

A few attestations had much higher inclusion distances (one with a 29 distance), those we are almost certain are due to beacon node latency.

Phase I Testing Summary

The efforts leading to the testnet spread across many months of planning, discussing, and envisioning a way to safely split an Ethereum validator key for decentralized ETH staking and increased infrastructure resilience on the Beacon Chain.

The best and brightest from the Ethereum community contributed their time and effort and the 12 testing partners were critically important to this pivotal moment of jointly running a distributed ETH validator and attesting and proposing together for the very first time.

We are all working towards a common goal of creating a more robust, and decentralized staking network for the coming merge and future of Ethereum. The feeling was that a meaningful step was taken towards achieving that goal as our KPIs were exceeded and we look forward to extending testing parameters in Phase 2.

Phase 2 Testing

Phase 1 taught us a lot, setting the scene for a much more involved Phase 2. We like the concept of iterating fast and testing early, especially with something like the SSV project as we can risk breaking stuff on testnet.

Phase 2 testing scope is still TBD but will be focused around a long-standing testnet allowing anyone to join as an operator and spin up an SSV node and allowing anyone to select SSV nodes from that network, split their validator key into secure shares, and run an SSV Pyrmont validator. This in itself feels like a huge leap forward.

Phase 2 is expected to include basic web UI for ease of use, and anyone with existing Pyrmont validators will be able to import them and select from a list of operators how to split their stake. Unlike Phase I, key distribution will be smart contract based and trustless.

The plan is to publicly launch the Phase 2 testnet in June! All are welcome to join as operators or users, the more the merrier to stress test the system and help identify any issues that may appear at scale.

In the meantime, you can look forward to additional educational resources on SSV to be released in the coming weeks, take the opportunity to spin up a local SSV network (see how here) and visit the EF SSV discord channel or SSV discord to join in on the discussion. We look forward to seeing this community grow and to deliver an even more robust Ethereum network with enhanced staking options for all types of users and needs.