The Security Kitchen

In the night between September 7th and September 8th, around midnight , the Colt network in Europe went black.  No, it wasn’t a little hickup, the network just went down.  In the blink of an eye, their top-notch multi-datacenter, redundant cross-channel lines was degraded to zero. It didn’t last for an hour though, it lasted over 16 hours ! At this moment I can find no confirmation that the whole situation has been resolved and all services have returned to normal.

From outside (at least from my ADSL line) it looked like the whole network was vanished.  All traffic just stopped at BNIX (which is the Belgian Internet Exchange).

Belgium, Switzerland and UK were among the affected countries, but nobody actually knows what happened … and that’s the worst part of this story.  After a few hours into the working day, the support center switched to a recorded message saying they had a major outage and were doing everything to resolve it.  The only online report on the outage was one by The Register.

Meanwhile, speculation started.  Some blamed it on a severed underseas cable, some on a software update that went bad. And then there were the rumors about a DDoS attack on the Colt network.  The fact is, NOBODY KNOWS !

As a company, Colt should have been much more open in their communication.  By not letting your customers know that you are on top of the issue, that you know what is going on with your network and how to resolve this kind of issues, you have seriously damaged the trust in your service.

Sure, I know that serious claims are being written as we speak and this outage is gonna cost you a shitload of money but that shouldn’t prevent you from communicating openly with your customers.

I’ll post an update when more details emerge … right now I think we have found one more provider NOT to consider for our critical network projects…

Update 1 :

Apparently, Colt has added a message to the frontpage of their homepage (see the picture above).  Just more fog …

1. Armorguy says:

   September 9, 2009 at 1:13 am

   Has nobody learned from Apache? Full, timely, and relevant disclosure when problems come up goes a long way to keeping your customers from getting overly angry. Apache did great on this and, for the most part, been forgiven for their problem Colt, it seems, is looking for epic fail.

   Sheesh.

   Like or Dislike: 3  0
2. admin says:

   September 9, 2009 at 1:21 am

   Exactly my thoughts. I personally think Apache has raised the bar handling the breach the way they did.

   Like or Dislike: 0  0