Latest Announcements from the Forums

2008-04-11: Unexpected Downtime
The datacenter that houses Arclight (the xepher.net server) went down for several hours early this morning. It seems to be back up now, but it was up and down quite a bit, so I'm not counting any chickens before they're hatched. Anyway, not much I can do about it... I noted that uptime had gone from 43 days to 15 minutes a few hours ago, then I lost all contact with the system. Finally got a support request through and got the server restored, but it's back to 10 minutes uptime. I think they had some fairly major problems they're not really wanting to tell us about. Either that, or just "coincidentally" the xepher.net server went down twice in 4 hours, with a report from the datacenter of a "brief outage" right in the middle of that timeframe. Considering the server didn't even go completely down when a hard drive failed, I think it's much more likely their power grid reset a few times tonight and they just don't want to admit it.

Anyway, shouldn't be a big deal, just letting everyone know why things were unavailable for a few hours. ...
[Read More]


2008-02-19: Short Downtime On Thursday (Now Completed!)
Update: The new drive is in and working. Things'll be a tad slow for about two hours or so while it rebuilds the array, but after that we're back to full data integrity and performance.

The server is getting a failed drive replaced on Thursday. The replacement should take about 30 minutes, and occur sometime in the afternoon or early evening (US CST) of February 21st. Shouldn't be a big deal, and most people probably won't even notice, but just wanted to let everyone know in advance. If something goes wrong, or you can't connect for more than an hour, feel free to ask me for a status update via AIM. My screen name there is Xepher42. ...
[Read More]


2008-02-09: Possible Problems
As was previously announced, the datacenter which hosts the xepher.net server went offline for building power repairs this morning. Things came back up this afternoon, as per the plan. A problem seems to have occured with one of the drives in Arclight (the server behind xepher.net) and it's refusing to talk to the machine anymore. Thankfully I had the foresight to build the system on a raid5 array, meaning the loss of a single disk doesn't actually result in data loss. That's why you're still able to read this post here, because everything is still working. Normally, I'd do some troubleshooting, and probably end up getting a new drive shipped to the datacenter and installed. Problem is, I go on a week-long trip tomorrow morning. Thus, that's gonna have to wait until I get back. While the remaining drives should be just fine and keep everything running until I can replace the bad one, I don't like having no redundancy. I'm currently trying to backup all data to my desktop here, but it's a lot of stuff, and my cable modem is relatively slow. Hopefully it can grab everything before I leave tomorrow, but whether it does or not, I'd still suggest everyone make sure they have their own data backed up, just in case. If it decided to drop ANOTHER drive towards the end of the week, my backup would be several days out of date too.

Anyway, I don't want anyone to panic (at least not as much as I am) but I just wanted to post that in case the worst happens, and you wonder why the server is down. Really, the chances of another drive failure are pretty slim, especially considering they survived whatever hurt this one. Heck, the old server ran on just 2 drives for 4+ years with no problems! I just tend to be overly cautious and concerned sometimes. I'll be back next sunday, the 17th, and hopefully I can get a new drive shipped to the DC and installed shortly afterwards. I'm gonna just look at it this way fo ...
[Read More]


2008-02-08: Borked Email
Two days ago I updated a lot of the software on the server to close a couple of security bugs/notices they announced. During the update however, we had two small glitches. One, was that I managed to disable CGI for the webserver, meaning if you ran perl scripts or such (like movable type) Reinder of ROCR pointed that out to me and I got it fixed yesterday. Today I got up, and had no email... but a couple IMs from people saying they had no email either. (Unfortunately, I was already late for work, and spent the next 6 hours on a job site.) Turns out I'd updated a package I didn't even intend to... the spam filter. The new version changed a couple variable names, so it wasn't able to run, meaning mail just got queued up, and after a while it stopped taking delivery, asking the sending servers to queue it up there and retry later. I started fixing it, and as soon as it was able to run, it delivered all the queued emails... but I hadn't finished fixing the config yet, so it delivered everything as though it were spam/junk mail. Anyway, long story short, no mail should've been lost. About 100 messages were delivered to various Junk folders though, so you should all dig through the past day and a half of those to see if one or two were legit. The rest of the mail that didn't get delivered should be queued up on other mail servers and begin trickling in over the next few hours now that it's accepting delivery again. The update occurred just after 0600 UTC on the 7th. That's midnight, the start of the 7th CST here in the states, so you only need to look back through the Junk folder until about then.

Sorry for the trouble on this one guys. I botched it, as when I told it to do the update, I didn't notice it was planning on updating my spam filter package. If I'd seen that, I'd have directly tested it right then and there. ...
[Read More]


2008-02-06: Datacenter Downtime And Xepher's Going On Vacation
I was just informed by the datacenter who houses the xepher.net server that several power transformers servicing their building are being replaced this Saturday. They estimate 3 to 5 hours downtime starting at 8am CST on Saturday the 9th of February. I don't know if that just means routers are going down, or if all the servers are gonna lose power as well. We may get to see how well I built the redundancy into this new machine sooner than I wished. If for some reason it takes longer than the estimate, or you just want status updates, feel free to IM me. Xepher42 on any of AIM, Yahoo, or GoogleIM. Though note that I have no intention of being up before noon on a weekend. :-)

Additionally, I'm taking a week-long ski trip on the 10th through the 17th. I'll be taking my laptop with me, but I probably won't be online much. (I'd obviously rather be skiing. :-) ) Shouldn't be a big deal, but just figured I'd let ya'll know not to expect much response from me during that week. ...
[Read More]

All code, images, and content in this page are licensed under a
Creative Commons Attribution-ShareAlike 2.5 License.