Websites Outage - 22nd September 2020

We are currently investigating problems, since approx 10.45am, with one our servers located at the Cheltenham data centre. This server, server159.eqwebs.com, controls almost all of the websites we provide to clients.

Server159.eqwebs.com was re-booted at approximately 10.40am to complete the installation of some automatic sofware updates. Although the server restarted after the re-boot it would not then display web pages


UPDATES

Despite being able to start the web server program, via direct access to the server, it still fails to display web pages

10.50am engineers at the data centre start to restore the server from a backup taken at 5am this morning

17.03pm Despite having restored the server from 4 different backups during the day we have been unable to get any of the websites working. The server is now being restored to the last successful backup (5am this moning). This will be used to migrate sites to a new server, which will be on line tomorrow morning. The migration of websites will take several hours, so it is likely to be sometime tomorrow afternoon before websites are visible. When the websites are initially visible they will not be user editable until users have been set-up on the server and the appropriate file permissions changed.

23rd September 8.30am The original server is now fully functional. After carrying out investigations overnight it has been found possible to get the original server working again. A full investigation of the cause of the breakdown will be carried out this morning.

10.24am It appears that an incorrect firewall zone was probably set to default and was activated at re-boot. This blocked all external services except SSH. Firewall rules from the correct zone have been copied to the other zone, in case the firewall defaults to the wrong zone again.