2013 Cartell IT QOS Report

2013
Uptime percentage (excluding scheduled maintenance) 99.995%
Uptime percentage (including scheduled maintenance) 99.995%
Scheduled downtime 0 min
Unscheduled downtime 25m

Cartell.ie outage 2013-08-12

Between 16:03 and 16:33 the Cartell.ie IP Address 78.153.204.100 was for the most part unreachable to IP traffic. This was caused by a firewall crash (that auto restarted) and lead to a short break in communication between components in our HA Cluster. The loss of communcation between nodes triggered an automatic failover to a standby system which served requests for several seconds. Since all systems were operational, the failover caused a ‘split-brain’ situation that was immediately and automatically identified, and the standby system was stood down. However, in standing down, arp cache tables in the data centre we not updated to route IP traffic back to the live system. It took the IT team over 25 minutes to identify that the live system simply needed to re-broadcast its availability to the network. As a result of this outage, our uptime for 2013 has dropped from 100% to 99.9943% (YTD).

The Cartell IT team are extremely disappointed and wish to apologise to our customers for any inconvenience.