Source: Parilov - Shutterstock

• After the crash of the Medalla testnet, Prysmatic Labs has clarified that the launch of Ethereum 2.0 will not be delayed due to the bug in the Prysm client.
  
• After the network participation rate on the Medalla testnet crashed to below 2% on August 15th, it increased again yesterday to over 68%

As reported by CNF, the final multi-client testnet for Ethereum 2.0 suffered a setback a few weeks ago when a bug caused non-synchronous nodes to appear and the Medalla network almost crashed as a result. Although the bug, which only affected the Prysm client from Prysmatic Labs, has already been fixed, concerns have been raised in the crypto community that this could delay the start of Ethereum 2.0 phase 0.

Prysmatic Labs has now tried to dispel this rumor via a blog post describing the exact cause of the bug. Raul Jordan, editor of Prysmatic Labs wrote that the schedule for the introduction of ETH 2.0 remains on track despite the mishap. According to him, he continues to support the launch of mainnet two to three months after the launch of Medalla, as originally planned:

We believe this incident does not inherently affect the launch date. The Prysmatic Labs team recommends ETH2 launch schedule to continue with no delay. The incident from this weekend was a good stress test for many clients and actually checks off a few requirements on the launch checklist. While the launch date has not been set, we believe the expected launch target of 2 to 3 months from Medalla genesis is still an ideal timeline.

As Jordan elaborated, the failure was the best thing that could happen to the testnet. So all the experience from the incident will serve to prevent the problem in the future. In addition, all lessons learned will be placed on the public checklist of requirements for an Eth2 deployment to ensure the resilience, security and proper release of clients.

Jordan also stressed that the testnet “is not dead”. As long as it is possible to operate a node and as long as validators can validate, the testnet can be fully functional again at any time. On August 19, Jordan stated in this regard:

We still have hope. Participation has now climbed from 0–5% to 40%. The chain needs > 66% to finalize.

The data from BeaconScan currently shows that, after slipping to 1.63% on August 15th, the participation rate was back at 68.59% yesterday, above the important 66%. The Medalla testnet thus appears to be stabilizing again, while blocks are being validated again.

Source: https://beaconscan.com/stat/networkparticipation

What caused the crash of the Ethereum 2.0 testnet?

As Jordan explains, the problem started with receiving bad responses from 6 different time-servers that repelled most of the nodes running the Prysm client simultaneously. Subsequently, the Prysmatic Labs team tried to find a quick solution, but it contained a critical bug that “removed all the functionality necessary for our nodes to function”.

This problem resulted in network partitions where everyone was synchronizing the blockchain at the same time, but could not find a healthy peer. Ultimately, it was only the ill-considered bug fix that caused the big problem, which nearly paralyzed the network:

Worried by the urgency of the original problem, we didn’t think too much about all the implications of a potential fix, and focused more on quickly releasing it than checking carefully if it would break anything else in our nodes. […] Even worse, the roughtime bug had fully recovered by now, which would have likely fixed the issues in the network had we not acted so swiftly.

To prevent the problem from reoccurring, Prysmatic Labs will make the following changes to the Prysm client:

Starting from this incident, we will rely on system time only. If a validator’s time is indeed off, we can tell them, but will not forcefully change it. Other eth2 client implementations only use system time and we will too.