chains of events and upside down servers

Tonight we had a series of about 12 power cuts within 30 minutes or so. Each was short (some only a few seconds) but they lead to a time consuming series of events.

First, I went downstairs and turned off all the servers to save them from the messy power supply (yes I know we need a UPS but …)

When the power seemed to be back on to stay I went to check things. The internet connection was unreliable, so I power-cycled the router. That improved things only a little. So I started looking at the IPCOP interface (our firewall), nothing obvious wrong but I noticed that I had now downloaded and installed the latest update. So I did that.

That install mentioned a command to type at the command line, so I went to SSH in, but it would not accept the password. Strange. Checked settings, tried again, still no good.

So I went to the IPCOP box in the rack cabinet and plugged in a keyboard and monitor (1st time since we moved here just over a year ago). Still would not work. I wondered about re-installing and that is when I noticed that the IPCOP machine was in the rack upside down. Oop’s. Still it has had a best uptime of 179 days (another powercut stopped it at that point) so it can’t be too bad.

I then booted as a single user, changed the password and got access again at which time everything was hunk dory.

The server is now the right way up :-)

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>