Monday, December 10, 2012

Mail Server Failure

At approximately 4:30am PST our mail server suffered an unexpected failure.  We have isolated the problem to a corruption of the mail database.  We're taking steps at present to recover the database from the available log files.  By 11am we will know if that process was successful.  If so, we will be fully operational at that point.  If our database fails to recover, we'll initiate a full restore from last evening's backup.  That could take up to two more hours to process meaning, worst-case, we will be back online at 1:00pm PST.

Sorry for the inconvenience.

-Alex

----

Update: Our attempt to run the recovery utility is being hindered by a lack of sufficient drive space.  We're exploring the possibility of running the recovery process on a different system and then moving the repaired database back to our mail server.  I'm still confident in our 1:00pm PST repair time.

We also discovered that our latest backup is from Friday evening rather than Sunday evening.  I still hold full database recovery as an option but I'm going to try to avoid it as we would lose all the mail that came in over the weekend.

-Alex

----

Update: Chris successfully got  Daryl to act as a surrogate recovery server.  We're waiting for results of the process now.

-Alex

----

Update: Take a few moments to go through outlook and your phone. Make a note of any important messages that you received over the weekend.  There is a chance that the recovery process may delete some of those messages from your inbox.

----

Update: The recovery process is chugging along but it is taking dramatically longer than I expected.  At present speed I'm estimating that we won't be functional until tomorrow at noon. At this point there is a choice:

1) continue the recovery process, recover all mail, but have limited access to new mail until tomorrow at noon.
2) restore from the Friday night backup, lose all mail between Friday at 8pm and Monday at 4am but have full access to new mail by 2pm today.

feel free to email me with your thoughts but I'm going to make this decision in the next few minutes.

-Alex

----

Update: I've decided that the productivity lost today is more valuable than the minimal number of messages we may have received over the weekend.  I asked Chris to cancel the database repair process and I'm recovering from our Friday evening backup. ****YOU WILL LOSE MESSAGES THAT CAME IN AFTER FRIDAY AT 8:00PM**** Right now, you should go into your phone, ipad, or outlook and make note of any important messages you received over the weekend.  I estimate that we'll be live again at 2pm today.

----

Update: Estimated recovery time now 2:30pm today.  Accuracy confidence is 90%.  The data recovery job had a slight hiccup but seems to be going smoothly now. 

----

Update: Mail server is now functional and stepping through it's queue of cached messages.  I have successfully sent and received mail and can confirm that we are fully operational.  There will be a rescan of your inbox at 2:30 to remove all the spam that is currently bombarding your inbox.  If necessary I'll do another rescan at 3:00.

Again, sorry for the inconvenience.  Be aware that messages sent to you between Friday at 8:00pm and Monday at 4:00am are most likely lost.  If you expected a message in that time frame, let your constituents know about the failure.  We have an option to recover that data if it becomes necessary but it is a lengthy process. If you believe there may be crucial emails in that window, please let me know.

-Alex

27 comments:

  1. From this blog you will get al the info that you need on how to write essay in ASA format. I think that this could be really helpful for your future works

    ReplyDelete
  2. Your blog inspires me every time when I read it. I love reading blogs. I don't excite always by reading blog posts. But in your blog I find something that implies in blogging. Anyway, you're unquestionably a great blogger and you have a skill to make your reader to keep reading your blog again and again. Keep up the magnificent work. Continue moving the people! This is one of the very interesting post. Because I am new in blogging and I require great instructional exercise like your post.All the best, best online essay writing services provides writing help for those who seek help from them.

    ReplyDelete
  3. If you are getting issues while connecting your Hp printer with your desktop then it might be because of compatibility issue. it will ask for HP driver update to make the connection. you need to update your drivers or install the latest drivers.

    ReplyDelete

  4. If you are attached to using keen devices like cell phones, tablet PCs, workstations, and different other electronic devices then you might have seen that we purposely need our email accounts to take advantage of our devices.
    Comcast Email Support
    <a href="https://contactformailsupport.com/comcast-customer-service>Comcast Customer Service</a>

    ReplyDelete
  5. Are you out of time and require quality and Essay Writer Services?. We offer Information Technology Writing Services to scholars, and we guarantee that all scholars can afford our high-quality services and products.

    ReplyDelete
  6. This blog is very informative and useful . Thanx for sharing with us.
    https://wwwmcafeeactivate.co/

    ReplyDelete

  7. Thanks for the informative article. This is one of the best resources I have found in quite some time. Nicely written and great info, I really thank you for sharing it.
    For any query regarding for this office setup and activation Some details, Just Visit My website ...


    ReplyDelete
  8. Thanks for sharing this helpful information, you have nicely explained about email and their problems. And if anyone feels any help regarding quickbooks tech support please visit our site.

    ReplyDelete
  9. First You got a great blog.I will be interested in more similar topics. I see you got really very useful topics, I will be always checking your blog thanks.
    Note for the visitor: - If anyone needs home appliances installation or repair service please visit geek squad


    ReplyDelete
  10. This comment has been removed by the author.

    ReplyDelete
  11. You discussed really valid and good points in your post and It really happened some person do not have the right idea. But with your awesome tips, one can easily identify it.
    .If you have any query regarding McAfee setup then get in contact by contact number or macfee support

    ReplyDelete
  12. If you are facing any problem in your laptop and need some technical assistance then you can contact Geek squad Support for help. they are providing the best service in the USA and UK. you can contact them through helpline no. or you can chat with an agent.
    geek squad tech support is a Best support to your appliances our expert teams Ready to always help you 24x7.

    ReplyDelete
  13. You discussed really valid and good points in your post and It really happened some person do not have the right idea. But with your awesome tips, one can easily identify it.

    If you are interested in McAfee activate
    , let's connect to discuss the opportunities.

    ReplyDelete
  14. Your article brings a lot of valuable information. bloxorz

    ReplyDelete
  15. You should take some time and evaluate the Literature Review Writing Service and the benefits that accrue to the clients. Most students buy Cheap Custom Term Papers and get ripped off.

    ReplyDelete
  16. Nice information provided by you, it really helped me to understand about the mail server failure issue. I have also referred this article to my friends and they also liked this informative post. We have also faced this issue with my business webroot geek squad tech support

    ReplyDelete


  17. Geek Squad Tech Support provides the best technical help and guidance. Get in touch with extensively trained tech support team for all sort of queries and issues regarding your devices. You can contact Geek Squad round the clock according to your convenience.

    ReplyDelete
  18. Geek Squad Tech Support is a team of experts that provides tech support services for various devices at home or office. For any tech help contact our experts at Geek squad tech support number.

    ReplyDelete
  19. The accounting of Master Thesis Literature Review understand the need to craft an exceptional personal statement. A good Paper Writing Services Legitimate can make your Legitimate Essay Writing Services while a bad one can break the same.

    ReplyDelete
  20. There are a very interesting question and very positive thinking in
    geometry dashyour mind so I like this topic so I one suggestion we can use it now bing

    ReplyDelete
  21. Get online assignment help cheap by expert writers to secure good grades, Allassignmenthelp provides professional assignment help in various subjects for students studying in colleges and universities across UK, USA, New Zealand, and Canada. We deliver quality solutions for assignments in over 100 subjects. Our customer support department is online by expert writers online 24/7 Feel free to contact.

    ReplyDelete
  22. This comment has been removed by the author.

    ReplyDelete
  23. Are you facing some interruption in brother printer with the limelight of error code? Well, underlying of error code 2147500037 disturbs a customer a lot and they cannot take decision what to do or not. Feel free to contact our expert.

    ReplyDelete
  24. If your want any query related to tickets and want to take advantage of flying at cheaper fares, then call Southwest customer service team. Here you can grab amazing deals and discounts with every booking.

    ReplyDelete
  25. With the ample advantages and opportunities to make your journey worth it through Lufthansa Phone Number, grab amazing deals and discounts here on your booking.

    ReplyDelete
  26. Want any help regarding ticket booking or want great discounts and deals on air-tickets. If yes than reach Lufthansa Customer Service anytime. Fly across the seas with your best travel partner.

    ReplyDelete
  27. Our experts at Canon Printer Support are so skillful that they don’t leave any issue unresolved. Also, the experts are at your service 24/7 without paying any heed to the issues apart from work.

    ReplyDelete