Post-Mortem: September 10th, 2025 Slashing Incident
A detailed postmortem on the September 10, 2025, slashing incident. After in-depth investigations, factors were external, and users are safe
On Wednesday, September 10th, 2025, the SSV Labs monitoring systems detected two distinct slashing incidents. The first affected a single validator, and ~1.5 hours later, a second incident impacted a cluster of 39 validators. Both incidents were immediately escalated, investigated in depth, and confirmed to be external to the SSV protocol.
Below is a detailed timeline of events, along with conclusions and lessons learned.
Key Takeaways
These incidents highlight several important lessons:
- Validator Key Management: Keys must remain confined to a single, trusted environment. Running validators in parallel—even unintentionally—creates a high risk of double-signing and slashing.
- Single Cluster Instance: Validator operators must ensure only one instance of a cluster is active at a time. Redundancy should be managed within DVT, not through duplicate environments.
- Slashing Protection: Slashing protection is critical, especially during maintenance, migrations, or failovers. SSV provides strong safeguards when keys are managed exclusively within its infrastructure, but protections weaken if keys are reused externally.
By design, SSV reduces slashing risk by distributing responsibilities across operators. However, if validator keys are run outside SSV, the guarantees no longer apply.
Current Status
- SSV protocol and infrastructure remain uncompromised.
- No changes are required from SSV operators or stakers at this time.
- Both incidents were caused by external operational errors in key management, not by any issue in the SSV protocol.
Incident #1: Single Validator
Timeline (UTC):
- 11:51 – First slashing event detected.
- 11:53 – Monitoring alert triggered.
- 12:00 – Alert confirmed as legitimate.
- 12:01 – Began gathering all available data.
- 12:09–15:14 – Logs received from operators (6 total).
Findings:
Beaconcha.in data confirmed that the validator was slashed due to a double-signing violation (two attestations for the same epoch). However, cross-checking SSV Node logs and telemetry revealed only a single attestation signature. Since the SSV Node reliably logs every attestation it submits, there is no evidence that the double-signing originated from SSV infrastructure.
The SSV Labs team and SSV DAO are also in direct contact with the staker of the affected validator. Their input is critical, and we depend on more details from them to further advance the investigation.
Conclusion:
This slashing event was not caused by SSV Nodes, but rather by factors external to the protocol. Further investigation with the validator’s staker is ongoing.
Incident #2: 39 Validators (Cluster)
Timeline (UTC):
- 13:17 – Second wave of slashing detected.
- 13:25 – Confirmed all slashed validators belonged to a single SSV cluster (Operator IDs 14, 17, 22, 71).
- 13:38 – Contacted validator owner (Ankr).
- 13:45 – Ankr acknowledged and immediately shut down all affected operators.
- 14:42 – Logs received, consistent with Incident #1: no evidence of SSV double-signing.
- 15:44 – Ankr confirmed the root cause: an internal maintenance mistake that ran a parallel validator instance outside of SSV.
Conclusion:
The slashing was not protocol-related. It stemmed from validator keys being simultaneously active in two different infrastructures—an operational misconfiguration during maintenance. Ankr has since confirmed it was related to their internal key management practices.
Partner & Community Cooperation
We thank our partners, including Ankr, for their swift collaboration and transparency, as well as the community for constructive engagement throughout this incident. Open cooperation is vital for network resilience.
Please reach out on the SSV Discord if you have any questions.
