RubyForge down

06 Oct 2006

If you're here, it's probably because RubyForge is inaccessible and you Googled "rubyforge down". Until it comes back online, you can still get files from the Bytemark file mirror, and you can get gems from the Bytemark mirror using:
gem install --source a_gem

9/28/09: 0610 EDT: RubyForge main site offline. 0930: Troubleshooting; it's a database problem. Restarted database, all seems well now. Still investigating original problem.

6/10/09: 1230 AM EDT: RubyForge down. 0902 EDT: I wake up, investigating, RubyForge is unreachable via ssh/http... but I can ping it. Ticket filed with colo. 1009: Back online. smartd checks normal. Still investigating cause of downtime.

4/3/09: 1251 EDT: is down _again_. See notes below on mirrors and whatnot.

4/1/09: 1649 EDT: It seems that is having DNS issues; this is leading to all the subdomains (e.g, being inaccessible. 1833: Seems to be resolved now.

10/3/07: RubyForge went offline at 2112 this evening. 2224: Filed a support req with our colo provider. 2239: They will attempt a remote restart. 2151: Back online.

9/12/07: RubyForge went offline at 0528 this morning. 0650: Filed a support req with our colo provider. 0727: They wouldn't be able to hook up a IPKVM to it until 0900, so I just asked for them to reboot it. 0751: Back online.

1/31/07: RubyForge went down at 0841. It's running out of memory and I'm competing with the OOM killer to get enough of a CPU slice to kill off some processes. 1007, back online. I'm going to switch to Apache's prefork MPM to see if that can limit memory usage a bit.

1/28/07: RubyForge went down at 0115 due to a kernel panic which occurred as a result of it running out of memory. Back online as of 0159 PM.

1/18/07: RubyForge went down at 0433 AM. Crash was due to a kernel panic. Back online as of 0955.

12/14/06: RubyForge was down from 1332 to 1425 PM and again from 1703 until 2052. This appears to have been caused by bots hitting a particular ViewVC function and triggering a bunch of diffs. Diffs have thus been disabled.

11/20/06: RubyForge was down from 0003 to 0830 last night; seems like the cronjob problem is not yet sorted out. Thanks to Dennis for restarting things. Back online now.

11/19/06: Our hosting service will be installing some new hardware and thus RubyForge may be offline for 10-15 minutes (hopefully) around 3 PM EST today. Update: Total downtime was around 7 minutes, back online now.

11/18/06: RubyForge was down from about 2 AM to 0730 today; Apache seems to have shut down around 2 AM. There were a bunch of sed processes sitting around so it looks like a crawler maybe have hit a bug in ViewVC. I'll ask around.

11/16/06: RubyForge was down from about midnight to 0800 today due to a mistake I made in a cronjob. It's back online now.

10/6/06: RubyForge is currently offline. We're working with the colo facility to figure out what's going on. I'll post more details when I have them. Update 11:58 AM: In the meantime, you can manually get files from the Bytemark mirror. Update 12:25 PM: The machine has been powered up again; I'm logged in to it via a serial port console. Hardware/database all seems fine, yay ext3 and PostgreSQL! There's a problem connecting to the gateway, though, so it's still unreachable from the outside. Working with the hosting provider to resolve this. Update 1:10 PM: Network problem resolved, but after being online for 4-5 minutes the machine started reporting non-maskable interrupt errors. We're heading out to the colo facility to pull the box and ensure we don't have a dead fan or something. Update 3:28 PM: Back from the colo facility, upgraded the RAID controller BIOS, all seems well. We'll keep an eye on it. Thanks for your patience!