Posts Tagged ‘outage’

Clickatell Outage

March 13th, 2012 1 comment

Clickatell have had an outage and have sent their clients an e-mail about it, the beef of which I paste here:

We regret to inform you that Clickatell’s system experienced a total outage due to complete power failure at the data centre which hosts Clickatell’s services. The outage occurred on Sunday the 11th of March 2012 from 09:50 GMT+2 and was resolved at 15:00 GMT+2. During this time all Clickatell services were unavailable. No messages were accepted for delivery and the system was unreachable.

Root Cause:
Clickatell’s services are hosted at a third party data centre. Electrical contractors caused a power outage throughout the data centre’s building while performing routine investigative maintenance on the UPS systems.

As part of their maintenance they bypassed the entire UPS system in order to safely work on it, and powered all infrastructure through one of their standby generators. During the maintenance this generator failed, and the entire building lost power. This meant both our live and standby systems lost power simultaneously.

Clickatell have never struck me as a small company and I was always under the impression that they are used by the largest corporations of the world, such as Google for example. In light of that, how on earth could they have allowed for such a thing to happen?