MDC problems — contributors please read

Last night we upgraded the MDC wikis to MediaWiki 1.6.5. Unfortunately we missed something in testing and the upgrades needed to be rolled back this morning. The result is that we have, at the moment, lost whatever changes were made to the MDC wikis between ~6:30p and ~10:30a Eastern time. We will endeavour to reapply those changes, but due to possible data corruption it may not be possible in all cases.

I apologize for any inconvenience this has caused, I can assure you that we’ll be reviewing our processes to make sure this doesn’t happen again. Thanks for your patience.

Update: we’re still noticing some issues and have thus locked the database for the time being. Current rollback is at almost a full day right now, but we will hopefully preserve the majority of yesterday’s changes by the time this is sorted out.

Update 2: We’ve managed to preserve the majority of data for all languages except Japanese. There is still a chance we’ll be able to get it all back, so in the meantime we’ll be leaving the MDC wikis as “read only” (ie: no changes to the db). The logic here is that we’d rather block changes for a few hours rather than lose changes if we manage to revert to a even-more-recent backup of the databases. Again, I’m very sorry for the inconvenience. As it stands all dbs are currently rolled back to their state as of ~6:30p Eastern yesterday, except Japanese which is back to ~4:30a yesterday.

Update 3 (and final): I’ve decided that, unfortunately, we’re going to accept last night’s wiki changes as lost in the main databases. We do have a record of those changes, however, and hopefully I’ll find some kind volunteers on the MDC mailing list to help out with reimplementing those. So, any changes to the MDC wikis made between 6:30p last night and 10:30a this morning (Eastern time) are no longer in the live databases.

As of now the MDC wikis are reopen for editing. Thanks to everyone for your patience, and I’m really sorry we messed this up.

Update (really final): Thanks to Victory, all “lost” data has been restored to the EN wiki. I would also like to thank Aravind, Oremj, Polvi, Nickolay, Callek, and Shaver for all being basically awesome and helping sort out yesterday’s mess.


2 thoughts on “MDC problems — contributors please read

Leave a Reply

Please log in using one of these methods to post your comment: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s