Note: We've turned publishing back on. If your site isn't coming up, please log-in and republish your site.
At about 11pm PST on Mon June 30th, one of the hard-drives on one of our storage systems broke down. This doesn't happen very often, but we're prepared to deal with the situation if it should arise. However, approximately 15 minutes later, another drive failed, causing one of the storage systems to go offline.
We've been dealing closely with the manufacturer of the device to bring it back online. In the meantime, we began restoring the affected sites from backup, as a temporary measure. As a result, some sites were unavailable from 11pm PST on June 30th until approximately 3pm PST, July 1st.
At 3:30pm PST on July 1st, about 30 minutes after we restored service to the affected sites, another hard drive failed on the device. Three failures from brand new, premium hard drives in such a short time-space is absolutely unheard of, but we've replaced the drive and expect to be fully operational by 7:45pm PST.
In the meantime, we've restored interim service to all Pro sites. Dan will be able to handle emergency requests to restore sites -- please email him at firstname.lastname@example.org if you need to have your site restored before 7:45pm.
We're as upset as you are about this monumental failure of hardware, which has been beyond our control. You can rest assured that none of your data has been lost, and we fully expect to restore service to all affected sites by 7:45pm PST.
If you have any questions or concerns, please email me at email@example.com and I will respond promptly.
UPDATE: During the restore process, another drive failed. We anticipate another 6 hours of recovery time for the affected sites due to this set back. If you have any immediate needs, please contact us and we can restore your site ASAP.
We are removing and completely replacing all of the original 15 Seagate hard drives from the affected storage system, and will be following up with the manufacturer with some serious questions.
Thanks again for your support while we work round-the-clock to restore service.
UPDATE 2: We've re-enabled publishing. If your site isn't coming back, you can fix it by logging in and republishing.