Prism Host - Chicago - Partial Network Outage – Incident details

Asia Pacific → Tokyo, Japan experiencing major outage

Chicago - Partial Network Outage

Resolved
Partial outage
Started about 2 months agoLasted about 20 hours

Affected

Datacenter - Chicago Illinois

Core Network Infrastructure

Updates
  • Resolved
    Resolved

    All impacted services have been operational for 14+ hours now, thus we're confident this issue has been fully resolved. We will be providing a detailed post-mortem following this incident later this week after internal discussion.

    This marks the most widespread/impactful outage we've ever had as a company, therefor we want to be as transparent as possible to ensure our clients are fully aware of why & how this happened as well as what we've done to prevent this from occurring again.

    We sincerely apologize for any and all inconveniences caused by this incident.

  • Monitoring
    Monitoring

    The majority of impacted services are operational at this time. We do still have a limited number of clients that are experiencing issues still. We are still working on this and will provide updates as soon as possible.

  • Update
    Update

    We are continuing to investigate route instability isolated to our Chicago location. The issue is stemming from communication inconsistencies between our edge routers at the spine layer. This is resulting in frequent route flapping and intermittent disruptions in traffic flow.

    Our network engineering team is working to re-balance route propagation and mitigate further impact. We'll continue to provide additional updates as soon as possible.

  • Identified
    Identified

    We are continuing to work on a fix for this incident.

  • Investigating
    Investigating

    We are currently investigating this incident. This is only affecting certain traffic landing on edge0.