User:Alessia/data centers on fire
Your data is burning!
OVH Cloud Datacenter Fire
(Strasbourg, France, March 2021)
A massive fire erupted at one of OVH's data centers, part of OVHcloud's network, which includes other 44 centers globally.
The servers housed in these centers hosted roughly 4 million websites, including specialised government platforms owned by agencies from France, Britain, Poland and the Ivory Coast. This event also affected a portion of the .FR web domain.
SBG1 and SBG2 went up in flames. SBG2, a five-storey, 500m2 data centre, and its contents were destroyed, while SBG1 was partially damaged. The building was eventually demolished. Two other buildings, SBG3 and SBG4, weren't damaged.
The different service branches that were affected by the fire were Web Cloud Universe (websites hosting), Bare Metal Universe (high performance servers for intensive computing, like media encoding, gaming, corporate security), Hosted Private Cloud Universe (private cloud infrastructure) and Public Cloud Universe (cloud service by OVH).
The exact cause of the fire remains unknown, but it appears to have originated from two UPS systems (devices that maintain server operations during power outages), one of these UPSs had gone through maintenance the day before the fire, during which some components were replaced.
OVHcloud nicely declined to comment on its fire safety protocol, but from the BEA-RI’s (France’s Bureau of Investigation and Analysis on industrial risks) findings we can tell maybe that was the problem, and not just that. The report from BEA highlighted issues such as the lack of an automatic fire extinguisher system in the building, and delay in electrical cutoff which made it more challenging to stop the flames. Toxic fumes from lead batteries also worsened the situation.
Initial investigation suggested that a water leak onto an inverter may have started the fire. So, the absence of an automatic fire extinguisher system, the delayed electricity cutoff, and the general building design likely worsened the damage to the buldings. However, the report still tell, curiously, that the exact cause of the fire remains uncertain. So these remain just hyphotesis.
How come just hyphotesis?
Strictly determining the cause of the fire could have had enormous financial consequences for OVH, given the ongoing legal actions from more than 140 clents, that asked for 10 milion euro in compensation, customers, so maybe it's better to keep a little bit of mystery to keep the boat afloat.
OVH's confusion regarding the location of their servers didn't improve their situation, leading some companies to believe their data was backed up on survived servers, as written on their contracts, when it was not. There were also issues with backup solutions, as some contracts inaccurately claimed that backup options were isolated from the server infrastructure. Great unluck here.
Despite suspicions that the water leak was the main issue involved in the 2021 fire, OVH continues to utilize water cooling for its servers. Eheh.
I was reading this news article, https://www.theregister.com/2022/09/13/ovh_sbg5_opens/, titled "OVH opens less flammable datacenter at site of 2021 fire" they could have easily put one ;) emoji there too.
In 2022, OVH reconstructed their datacenter building over the SBG2 rubbles. In the new SBG5 they made some changes to improve safety tho. They separated the data center from the supporting infrastructure. Batteries are now stored in fire resistant containers outside the data center, along with rooms containing transformers.
The entire facility is built with fire-rated walls and surrounded by double fire barriers. Very Early Smoke Detection Apparatuses are installed in all internal spaces, with both sprinklers and inert gas available to suppress the fire.
That was a good start, mainly for the investors, as it was a good way to introduce a new cold storage service called Cold Archive, one of four mini cute data centers builded across France. This service offers a long term storage for "cold data", a cold cemetery.
OVH?
Its name stands for “on vous héberge”, “we host you” in french.
OVH is Europe's biggest hosting provider and the third largest worldwide in terms of physical servers. It mainly serves clients in Europe and the US, and opened subsidiaries in SIngapore, Australia, India.
It was founded in 1999, by Octave Klaba and his family, in Roubaix, France.
OVH is a pioneer of free cooling, which uses water instead of air conditioning to cool processors down. This innovation has been the key for OVH's cost effective approach, helping them being competitive in the great sharks market dominated by giants as Google, Amazon and Microsoft. Quite interesting the new hybrid immersion cooling system, that uses thermally and not electrically conductive liquid, that claims to lower the carbon impact, use less space, being generally cheaper.
OVH markets itself as a green data center solution.
OVH relies in large part on nuclear power, for example the Gravelines data center sits next to the Gravelines nuclear power station.
During 2021 OVH joined the Climate Neutral Data Centre Pact: aiming for climate neutrality in data centers by 2030.
OVH also promotes itself as an independence solution for Europe.
Many european politicial nests would love to see OVHcloud succeeding as they see the still present US cloud dependency as a risk for the european sovereignty and economic competitiveness.
Klaba used these concerns to push for OVH as the favourite safe alternative, as its servers being outside the US are not subject to US search warrants and law enforcement, as well as intelligence agencies infiltrations, the latter could be a little bit weak as a claim.