Helium network was down for about 4 hours on the 11th of July due to a software update that affected validators, resulting in delayed transactions.
The team discovered an issue where Hotspots were communicating with both Mainnet and Testnet, which caused "a Hotspot to run out of memory and freeze/flatline", according to a report from the Network's status webpage.
During the outage, devices sending data over the network were not affected, but they halted miners' reward and token transfers. As a result, the team had to fix the problem by boycotting one block function in the blockchain, returning to normal operations.
Helium’s twitter status update disclosed yesterday, at about 10:20 a.m. EDT, that the Consensus Group had stopped producing blocks at block height 1435692 on the Helium (HNT) blockchain. As a result of poor network consensus, token transfers couldn't be completed, and new blocks could not be produced.
Helium is an IoT network that allows users to connect their devices from anywhere with radio coverage by using physical radio hotspots. On the Helium network, a Consensus Group comprises 43 validator nodes that are picked randomly at regular intervals to produce network consensus.
Helium engineers disclosed that there were two reasons why validators stopped producing consensus on the network.
First, the issue degenerated on the 8th of July due to a software update for validators. The v1.12.3 update was created to support the MOBILE token and the 5G Mobile subnetwork.
In the project's Discord channel, Digerati, a moderator for Helium, wrote that at the time of the outage, a large number of validators chosen at random to be part of the Consensus Group were running on the same Amazon Web Service (AWS) network, which was having technical.
The problem worsened because an auto-skip feature that should have auto-picked a new Consensus Group failed. According to the team;
The auto-skip feature didn't work as expected because of a known issue. But it is unclear what the team meant by the phrase "known issue.
Though the network couldn't choose a different Consensus Group, it was built to skip the blockchain by one block in order "to deal with situations like this," the team said in an update yesterday at 10:56 am ET.
Block production resumed by 1:45 pm EST with block number 1435693 being the first to be produced. The team said it worked closely with validator operators to ensure they were all in sync and then a new software update was released.
According to Coingecko, HNT has lost 4.1% in the last 24 hours and is now worth $8.76 at the time of this writing. Since its all-time high in November 2021, it has decreased by 84%.