Unexpected GnuCash Server Downtime: All Services Restored

Derek Atkins warlord at MIT.EDU
Thu Feb 24 14:44:30 EST 2011


Hi All,

You may (or may not) have noticed that the GnuCash Email, SVN,
etc. server was down since Tuesday.  The server should be back up now
(as of 13h14 US/EST) and all services should be running normally again.

Please let me know if you see any issues.

Failure breakdown: In short, the system lost a CPU fan (again!) and it
shut down due to overheating.  I replaced the fan and the system came up
again.

Long version: (Note that all times are in US EST (GMT-5)).

I know the server was still alive as of 11am on Tuesday when I was in
the Johannesburg airport and checking my email waiting for my flight.  I
know that it had shutdown sometime before 5pm EST on Wednesday when I
tried to access it from Newark airport, the next time I had Network
access.  (The timing of this failure was spectacularly convenient
considering I've been halfway around the world for the past three
weeks!)

When I finally got home at 11:30pm on Wednesday I noticed the system
completely turned off; I powered it on and it started to come up, but 10
minutes later the overheating alarm came on.  Considering my jetlagged,
exhausted self (and the annoying noise it was making) I decided to shut
the system down again overnight and deal with it this morning.

This morning I opened up the box and discovered that one of the CPU
cooling fans failed.  Strangely this was the same CPU that had a failed
fan in September.  This time, however, instead of burning out it had
thrown a baring.  Since this was the same model as the original I
decided I needed something better for this CPU so I started researching
better cooling fans and found one available from Fry's Electronics.  I
went out this morning, purchased the cooler, and then installed it into
the system.

The main system was powered up at 12:24 today and then all the VMs
slowly came back up.  Many of the VMs decided to FSCK due to the unclean
shutdowns and finally the last VM returned to service at 13:23.

I'm sorry if the downtime caused any inconveniences.  As I said, the
timing was as perfect as it could have been considering my travel
schedule this month!

-derek
-- 
       Derek Atkins, SB '93 MIT EE, SM '95 MIT Media Laboratory
       Member, MIT Student Information Processing Board  (SIPB)
       URL: http://web.mit.edu/warlord/    PP-ASEL-IA     N1NWH
       warlord at MIT.EDU                        PGP key available


More information about the gnucash-user mailing list