Cardano Developer IOG Explains Why 60% Of All Nodes Went Offline

The Cardano blockchain experienced an anomaly on Saturday evening (07:00 pm EST, between block 8300569 and 8300570) that caused close to 60% of all active nodes to go offline for a short period of time. While the blockchain did not come to a standstill, there were delays in transactions. But what happened and what was the cause?

One of the first to report the anomaly was Tom Stokes, co-founder and COO of Node Shark and operator of an ADA stake pool. He wrote: “A few hours ago over half of all Cardano nodes went offline. This is why decentralization matters,” and shared the chart below.

As the above chart shows, the network recovered to around 87% within a short period of time. However, the cause of the incident remained unknown for a rather long time. At press time, the inventor of Cardano, Charles Hoskinson, had also not yet commented.

Cardano Developer IOG Responds

A statement from the development company behind Cardano, Input Output Global (IOG), gave an explanation for the phenomenon. The company explained that a bug caused more than 50% of nodes to disconnect and restart.

“This affected relay nodes and block-producing nodes – edge nodes do not appear to have been affected,” said IOG, which went on to explain that this appears to have been triggered by a temporary anomaly that triggered one of two reactions in the node; some disconnected from a peer, others threw an exception and restarted.

According to the Cardano developer, this is a non-concerning event:

Such transient issues (even if they were to affect all nodes) were considered in the design of the cardano-node and consensus. The systems behaved exactly as expected.

Block production was only briefly impacted with a portion of the network falling out of sync for approximately https://cardanoscan.io/block/8300569 before nodes restarted.

As the company emphasizes, the impact was therefore rather minor – “akin to the delays that occur during normal operations and often seen at epoch boundaries.”

Most nodes recovered automatically without human intervention and performed a restart, depending on the choice of staking pool. Nevertheless, the company promises to further investigate the cause of the anomaly and to implement additional monitoring logging measures in addition to “regular” monitoring procedures.

Within the community, the incident sparked discussions about the network’s decentralization and whether the anomaly is worrisome. One user wrote that such an incident is not common and that it has never happened before on Cardano.

“There was one time where inefficient reward calculation over an epoch boundary slowed down nodes, that’s about it” the user explained. In general, however, the community agreed that the impact was minimal, as block creation only slowed down for a few minutes while nodes were rebooting.

Regarding the decentralization of the network, though, some users expressed concern. Some said the event revealed that Cardano’s node diversity is not good enough. The only reason not all nodes went down was because not all nodes accepted incoming connections. “The survivability here was not due to node diversity, but node robustness,” another user added.

Cardano (ADA) Price Today

The 1-day chart of ADA shows a breakout from a downtrend that has lasted more than eight months. The next major price target is the zone between $0.42 and $0.43, where former strong support which has turned into resistance meets the 200-day EMA. At press time, ADA traded at $0.374.