Resolved -
This incident has been resolved.
Aug 28, 00:10 UTC
Update -
Please be advised that the scheduled change planned for 19th August between 01:00 and 01:30 CEST to re-route traffic through the original connection has been cancelled. We will share a revised schedule as soon as a new change window is confirmed.
Aug 18, 19:42 UTC
Update -
We have scheduled a change for tomorrow, 19th August, between 01:00 and 01:30 CEST to re-route traffic through the original connection. A brief service disruption may occur at some point during this maintenance window.
We will monitor the services during and after the change to ensure stability.
Aug 18, 14:33 UTC
Monitoring -
A fix has been implemented and we are monitoring the results.
Aug 17, 01:44 UTC
Update -
While the network link between AWS and Equinix datacentre undergoing repairs, we have established an alternate network connection. Despite a smaller bandwidth, the new connection is already in a working state. Our team is working to brining Database services up, we expect Frankfurt SaaS to be available shortly.
Aug 17, 01:24 UTC
Update -
We are continuing to work on a fix for this issue.
Aug 17, 01:20 UTC
Update -
Direct Connect connections at at Equinix FR5 in Frankfurt remain impaired. AWS engineers have begun physically inspecting the impacted network equipment and working on this. We do not yet have a timeline for full resolution while we continue to assess the extent of impact to equipment.
Aug 16, 19:42 UTC
Update -
Direct Connect connections at this location remain impaired. The electrical isolation at Equinix FR5 in Frankfurt, DEU is now complete and AWS engineers have begun physically inspecting the impacted network equipment. We do not yet have a timeline for full resolution while we continue to assess the extent of impact to equipment.
Aug 16, 16:38 UTC
Update -
AWS connectivity to the Equinix FR5 storage facility in Frankfurt remains impacted. AWS Support is actively working to restore the network services. We will continue to monitor provider updates closely.
Aug 16, 14:03 UTC
Update -
AWS has confirmed that connectivity at the Equinix FR5 facility in Frankfurt remains impacted. AWS and its co-location partner have made progress toward restoring safe access to the affected area, and electrical isolation work is currently underway to de-energise the impacted infrastructure. Once the area is confirmed safe, engineers will inspect the affected network equipment to assess the extent of any damage and determine what replacements may be required.
At this stage, AWS does not expect a full recovery in the near term due to the potential scale of equipment impact. AWS connectivity services at this location remain impaired, and we will continue to monitor provider updates closely.
Aug 16, 10:51 UTC
Update -
Latest update from our datacentre partners in AWS Frankfurt. We are actively engaged with our co-location partner to restore connectivity at Equinix FR5 location in Frankfurt, DEU. Since our last update, we have made incremental progress to restore safe access to the affected area at the Equinix FR5 location in Frankfurt, DEU. In parallel, we have prioritized the order in which critical and high-priority racks will be restored, as part of the mitigation efforts. Based on our current assessment, full recovery is not expected in the near term and may extend over several hours.
Aug 16, 05:32 UTC
Update -
The issue identified now is related to network connectivity between storage and aws datacentres. The storage clusters have been restored; however, the network issue remains unresolved. We will continue to monitor the situation and provide updates as additional information becomes available. According to AWS, the next update is expected at 9:30 PM PDT.
Aug 16, 01:01 UTC
Update -
Datacentre engineers have restored services, and the storage centre is up and running. However, network connectivity has been impacted and work is under to restore connectivity.
Aug 15, 21:41 UTC
Update -
The storage vendor has notified that they are bringing the systems back online. We will continue to monitor this and bring Discover online as soon as the systems are fully restored.
Aug 15, 19:17 UTC
Update -
The storage vendor has advised that services are expected to be restored over the next few hours. The air conditioning and other cooling infrastructure have been damaged and are currently being repaired or replaced. At this stage, we do not anticipate any data loss, but we will be able to confirm further once the storage centre has been restored.
Aug 15, 16:28 UTC
Update -
The cooling system issue has been resolved; however, the temperature remains outside the safe operating range. We will proceed with restoring the environment once the temperature returns to the safe range.
Aug 15, 11:00 UTC
Update -
Datacentre engineers continue to work to resolve the power supply and cooling systems.
Aug 15, 09:55 UTC
Update -
The cooling system issue has been resolved. We will resume Discover SaaS services as soon as temperatures have stabilised.
Aug 15, 08:45 UTC
Update -
The cooling system issue has now been resolved. We will resume Discover SaaS services as soon as temperatures have stabilised.
Aug 15, 07:44 UTC
Update -
Datacenter teams are still working to restoring power and cooling services.
Aug 15, 06:45 UTC
Update -
Datacenter teams are still working to restoring power and cooling services
Aug 15, 05:46 UTC
Identified -
The issue has been identified and a fix is being implemented.
Aug 15, 04:46 UTC
