News Service Interuption

The article numbering server in our news cluster suffered a catastrophic failure a couple of hours ago.  All services have been restored on to the backup server but there may be some delay in new articles as the servers catch up.  It is possible that some articles were lost and cannot be re-fed into the system.

Update:  Due to an error promoting a reader slave (this is something we’ve only had to do once before, many years ago) to the article numbering master server, the article numbers being assigned to new articles were grossly incorrect.  As such, all articles received between when services were brought back online and the article numbering server was fixed, are lost to our readers.  I’m sorry we didn’t catch this sooner.

Update:  The news cluster continues to have some issues that we are working to iron out.  The replacement article numbering server is having trouble keeping up with the feed and we’ve had to force it to catch up by flushing out its back log (loosing the articles) several times.  Please note that this only affects our overviews (article lists) and not our ability to retrieve articles by message-id from our spools.  We’re hope to have the situation stabilized soon.

-Kelsey

Leave a Reply

Your email address will not be published. Required fields are marked *

*