FUCKED: Server 1
Cross posted from http://winboards.net/viewtopic.php?p=119597#119597 :
In regards to the latest wave of problems on Server 1, here's the explanation. I'll update this post periodically to provide updates on this.
The cause of all the fail is DRIVE FAILURE!!!!!! - yay, right? no, not yay. I made changes to some of the epoll behavior to fix some of the forum/other PHP based reset errors, and turned off the global cache, which I forgot I turned off for exactly this reason. However, as far as downloads go, after some investigation there's massive amounts of failures in accessing an assload of blocks on that hard disk.
This very well may be an issue easily fixed by a good fsck, however to umount and fsck that drive will require a maintenance window and datacenter cooperation which is really not 100% feasible at this point. With the luck I've had lately this will probably not solve much of anything.
I'll look into this issue more later today.
In regards to the latest wave of problems on Server 1, here's the explanation. I'll update this post periodically to provide updates on this.
The cause of all the fail is DRIVE FAILURE!!!!!! - yay, right? no, not yay. I made changes to some of the epoll behavior to fix some of the forum/other PHP based reset errors, and turned off the global cache, which I forgot I turned off for exactly this reason. However, as far as downloads go, after some investigation there's massive amounts of failures in accessing an assload of blocks on that hard disk.
This very well may be an issue easily fixed by a good fsck, however to umount and fsck that drive will require a maintenance window and datacenter cooperation which is really not 100% feasible at this point. With the luck I've had lately this will probably not solve much of anything.
I'll look into this issue more later today.
Comments
I dunno how much I trust that not-very-smart assessment but I'll start taking stuff offline so I can umount that drive later today.
Thanks. I keep telling myself I'll put that box out of it's misery one day, I should probably eventually do it.
Yeah and I think even with the latest firmware it hates 1TB disks. Cherokee's epoll() handler spit out a 300MB log file of bad file descriptors in winworld's error log alone.
I have a KVM hooked up after steadfast fucked up and hooked it up to the wrong box at first, even with this image, so I should probably plan a time for this to go down for fsck. I'll handle this once my insomnia lately clears up.
I ended up installing a hacked up firmware since the latest BIOS didn't update the Sata controllers version to the right one.
Even so I still get random quirks here and there, but a decent PCI controller that has more than 2 ports is stupidly expensive.
I'll do it, just not this second. Also, I don't have an IDE disk, it will have to be SATA.
I think wdl2 only supported IDE but I can always have someone at work copy it for me... How many gigs are we talking at this point?
I have no idea really, the disk itself will be around 250 GB, I think most of it will be used.