Server Configuration Restored

I have just noticed that an important server configuration file got erased during the move. I have just restored it. It's quite amazing that the server worked at all without it :) There were several small things that broke because of the absence of this configuration file, most noticeably you might have noticed that strange backslashes (\) started to appear whenever you would put a single quite (') or a double quote (") in your Overstream's Author comments - and maybe also in just comments, I am not sure. This should be fixed now.

Server Optimization

Well, I can see at least everything is running - with some minor glitches, as reported by some users, but running! Over the next couple of weeks, I will try to figure out as much as possible how to optimize the server performance (especially with regards to the database), so I hope we can get past the glitches and the performance will be better!

Good Performance So Far

So far the server has been responding pretty quickly, but of course we need to wait a couple of days and see what is happening with it. Hopefully, this whole move will be worth it ;)

Alright, I am finally going to go to sleep (8:30 am here ;)). I'll ask my friend to keep an eye on our server just in case.

Enjoy!

Liftoff!!!

I think we have liftoff! At least I am seeing the site, and the videos seem to play on it :)

DNS Starting to Resolve

Some users are already reporting that they are seeing the new site.

If you see "Project Damage Control", it is due to a slight misconfiguration on the part of the hosting - I am on the phone to them right now about it. This will be easy to fix.

Instructed Hosting to Switch DNS!

OK, I figured there is only so much I can check without switching the DNS over. (For example, the logging into the site didn't work but I am pretty sure that was only because the domain of the site was not www.overstream.net).

So, I told the hosting to switch over the DNS! This means that once they have initiated the switch, we will just have to wait for the DNS to propagate - which could take 1-5 hrs depending on where you are. USA should be closer to 1 hr (because that's where the hosting is). In any case, keep checking the main site and at some point it should just come up.

Since you will probably see the site before I will, please post back here and tell me if something isn't working so that I am ready to fix it. Also, as I imagine there will be quite a stampede of users, this will be a real stress test for the young server ;) If anything doesn't work as expected, please tell me ASAP.

I will post when the hosting confirms that they have initiated the switch.

Thanks for sticking with me throughout this ordeal, and I hope we are almost there!

Currently Configuring Server...

Well I was hoping things would just magically start working, but it's not quite so easy - I have to make sure that the web server, the PHP and the database are configured in the same way as on the old server. Currently trying to figure out the necessary server options... this is a first for me :)

Got the Configuration Email!

Finally, I got the Welcome/Configuration email! In fact, I got it 35 minutes ago but it was stuck in my gmail Spam folder!

OK, now to figure out the status of the site. Once I verify that everything is working, I will instruct the hosting to switch over the DNS, and then we'll be in home stretch!

Files being copied RIGHT NOW

After about 1.5 hrs on hold, I finally got through to the hosting. They are copying the files over right now from the old shared account. To copy the files they had to shut down the account, which explained a "Page Not Found" you might have briefly seen on the site there. In any case, the tech claimed that the actual copying process will not take more than an hour. We are getting closer!

Account Still Being Transferred!

Eeek! Just got off the phone with the hosting techs, they said the account is STILL being transferred. Now I really don't see what should be taking so long to transfer the account, especially since I explained our situation previously and they agreed that they will try to do it as fast as possible.

In any case, the tech said that they were currently still working on moving the data over and promised that the account will definitely be transfered at most in the next 12 hours (which is really kind of a lot I think!). Well I am hoping their message about the transfer will come earlier than that. Guess it's a good thing I didn't stay up to wait for it to be transfered :)

Server Set Up, Account Being Transferred

Just got this from the hosting: "Your server has been setup and your account is in the process of being moved." So it shouldn't be long before I can start verifying whether things are working or not!

By the way, I got a server with a RAID (that's the thing that makes sure that your server keeps going even if its hard-drive fails - it just switches over to another one), and also with Managed Data Backups - this is when the hosting backs up all the data on the server to a safe location in case something drastic happens to the server, like a tsunami, earthquake, or a terrorist/hacker attack. :) I know some of you will be happy about this last option :)

Dedicated Server Installation

I have just called the dedicated hosting techs, and they told me that they got all the parts and the server will most likely be installed first thing in the morning (that's USA East Coast morning), so I should be getting the email with the info about it in around 6-8 hours. I also once again begged them to do anything possible to expedite the process, but apparently they don't do the actual installations during the night ;)

Once the server is installed, and the account transfered, I can start going through it to make sure that everything still works in the new place. I am hoping that this part will not take long, but as always we have to account for unforeseen difficulties. After I make sure that everything appears to work, I will give them the go-ahead to switch over the DNS, and after that it should only be 1-5 hrs for it to propagate (it was pretty fast last time, only took <2 hrs to propagate to Japan). So, things are moving right along.

Overstream Down For Maintenance

The hosting techs have noticed that we were bringing the server down again, but this time they gave me a chance to put my own maintenance page up. I linked from it to this blog.

The techs have promised to expedite the process of the dedicated server installation as much as possible. As soon as any ETA (that's Estimated Time to Arrival :)) will be known, I will post it. Originally I was told that "Typical installation times are 24-36 hours, but in your case we can do it significantly faster". Let us hope so.

Oh, and in case you are wondering how come none of my questions about the dedicated hosting were answered for 3 days straight (before today), the explanation is: "I apologize for the delay. Your ticket was mistakenly flagged to the wrong department." It's a curse, I am telling you... :)

Stressing Out The Shared Hosting

I can almost hear the shared hosting server creaking like a mast under a strong wind... I have a feeling they will probably suspend us again until we can make the transition to the dedicated server....

Unsuspended

And we are currently back up! Hopefully the surge of eager of overstream users will not force them to suspend the account again until we can transition to the dedicated service...

I will post more about the transition to the dedicated service and the downtime associated with that later to give an advance warning so that you plan on not using overstream during that time (I know, it's hard ;))

Waiting to be Unsuspended

Looks I like I have convinced the hosting to unsuspend the shared hosting account until I can move it to the dedicated server. They are taking a chance here - our account has screwed up the server enough times by now :)

Waiting to be unsuspended...

Dedicated Hosting Ordered

I just got off the phone, having ordered a dedicated server hosting. It will take them some time to set it up, and then to transfer the existing account and then for me to go through it and make sure that everything is working. They said that this might take a maximum of 24-36 hours.

At the same time I am trying to get the shared hosting to unsuspend the account for the time being (while the dedicated server is setup), but I can understand their reluctance - our account has a knack for bringing down the whole shared server (which has 100+ other accounts on it :)).

In any case, the bottom line is that in at most two days we will definitely have a dedicated server. Let's hope that somehow I can convince them to bring up the shared hosting account in the meanwhile... keeping my fingers crossed!

Account Suspended Again

Yep, I can see the hosting suspended the account again - due to all the traffic you guys have generated. And I was so hoping I would manage to calmly move to a dedicated server without any more downtime... :( But no, they wouldn't answer my emails about the dedicated server for 3 days ;)

Reaching Hosting Providers

I contacted several hosting providers about their dedicated hosting plans, but so far have gotten no response (this after 2-3 days). I tried calling our current hosting, but could not reach them (they put me on hold for ~5 minutes several times, and then said they were experiencing unusually high call times). This feels like a curse or something ;) Well maybe it was just the weekend, I will try more during the week.

If you guys have dedicated hosting recommendations, do tell!!!

Dedicated Server Move

It has come to the attention of our hosting that the server load due to overstream users is really getting waaaay too much for the shared hosting environment. So, we will be moving to a more expensive and higher-performing dedicated server account soon. I am looking at several dedicated server options right now.

Obviously, I will try to keep the downtime related to the move to a bare minimum. Also, I will try to schedule it when it's night everywhere in the world... :D I will post more info with exact information on when the move should happen. After this move we will have a faster and more reliable server, so that will be good :)

DNS Propagated (At Least for USA and Japan)

O joy! I can see http://www.overstream.net working again! So, the DNS has propagated to Japan already - faster than I expected. It is also working in USA, some users are reporting. So, we are well on the way to recovery! The hosting promises they will keep an eye on the account to see if it's doing anything funny, but I can't imagine why it would - it worked just perfectly on this server all this time.

I will be finally going to sleep (it is deeeep night here in Tokyo), and I will try to sort out any remaining issues tomorrow. I guess I feel tired in the same way as Bilbo from There and Back Again: for some obscure reason we were moved to another server, it didn't work out, and now we are back home. :)

Domains Pointed To The Original Server

Alright, the hosting technicians tell me that the move has been effected, and the domains have been pointed back to the original server. So, now we just wait for the DNS to propagate - some of you will see the site come up earlier than others, and I am crossing my fingers!

Since I am in Japan it will take me a bit longer to see the site come up.

DNS Propagation To Start Within The Hour

I just got word from a technician at our hosting that he is currently in process of moving the account back to the previous server, and that the DNS propagagation should start "within the hour". This means that the server should be accessible to the people in North America in about 2 hours or a bit more, and the move should be completed world-wide in about 6 hours. I will keep posting the details.

Maintenance Message Is Up

Finally, the hosting has managed to put the overstream maintenance message in place of the "suspended" message. We are proceeding with the account move back to the old server. I will report as soon as this move is made.

The last technician I have talked to has cited typical times of DNS propagation as 1-5 hours. The hosting is physically situated in USA in the East Coast, so for people in North America the DNS propagation time will be closer to the 1 hour mark.

Moving Back To Previous Server

Well, it looks like we'll be moving back to the previous server, since the hosting simply can't figure out why our account is causing such problems on the new server. Oh well, who wanted that new server anyway! I am keeping my fingers crossed that the move passes with no problems.

The site will be inoperative during the move - it might take DNS ~2-3 hours or a bit more to propagate. In the meanwhile I am trying to get the hosting to change that ugly "this account is suspended message" which is so confusing to a reasonable maintenance message.

Hosting Suspended Our Account Again

Overstream.net moved to the new server and is continuing to cause problems there for some reason (it should work, it worked just fine on the old server). This apparently is causing the hosting to "suspend" our account every now and then, so that all the visitors see when they go to overstream is an ugly "this account has been suspended" message.

I am trying to get through to them right now on the phone. Via email, we are also discussing the possibility of the account being moved back to the old server on which it has worked without problems. This move would entail shutting down the site for several hours while allowing the DNS to propagate, so that no user data is lost during this move.

And We Are Back Up!

And right after this informative call I can see that overstream.net is back up!

Hosting Says Still Working On The Server

Got through to the hosting once again, now they say that they have no accurate ETA on when they will finish the work on the server. The tech I got also couldn't tell me exactly what was being worked on either.

Hosting Says Server Back Up In 20 Minutes

Just got through to the hosting, they said that they are "currently working on this particular server, it should be back up within 20 minutes". Just our luck that we get moved to a glitchy server... ;)

Server Down Again

I can see overstream.net is down again. I have just called the hosting and got the message that "due to extraordinarily high volume of calls my call cannot be answered". So, it sounds like the hosting is having some major problem there.

Le Grand Content

I simply love this film!



And now we can see it with spanish subtitles! :)