Overstream Muse

Overstream Blog Moves

2008-11-01T00:45:00.000-07:00

The Overstream Blog is moving to this logical location :). Please change your bookmarks and feeds accordingly.

This blog is closing down, although it will remain here for historical purposes.

Future Plans

2008-09-22T11:11:00.000-07:00

[max] Previously, I have indicated that I be able to disengage from my day job around beginning of September, and (re)start Overstream development. However, it appears that this disengagement process (through no fault of mine, actually) is taking slightly longer than I have hoped, so I am still in the process of arranging that. I hope it won't take too much longer.

Meanwhile, we've gotten many emails from people proposing interesting new features and collaborations. It's clear that Overstream needs to expand to be able to implement the many awesome features you guys have requested and to offer more stable maintenance service. So, besides considering the technological development roadmaps, I will also be reading some business books and/or looking for a business person to join our team - with the goal being to set up a revenue stream (yes, currently there is none) which will give us the money to hire more people.

That's a lot of upcoming challenges, but I hope we will be able to conquer them all :)

Back from Vacation / YouTube Probs Fixed

2008-06-22T11:50:00.000-07:00

As many of you have noticed, Overstream had some problems playing YouTube videos for the last 3 days or so. Of course, this just had to happen when I was away :) Well, I am back now, and everything should be working again.

On Vacation Next Week

2008-03-20T21:10:00.000-07:00

I will be on vacation next week, getting some much needed rest. I have left a friend to admin Overstream and try to address any emergency issues that come up, but other than that most of your emails and comments will be answered when I get back.

The system seems to be fairly stable overall, and I certainly don't expect any problems. See you guys in a week!

Should Be Mostly Propagated

2008-03-12T08:33:00.000-07:00

The working overstream.net on the new server should be visible to most users now. If you still see "Under Maintenance", try closing the browser and opening a new one.

Everything seems to be working fine. If you notice any strange things, please tell me (e.g. by commenting here).

By the way, the previous server had a problem with sending emails (because of the broken clock a lot of its emails would get thrown out by the mail routers). If you registered within the last week but still haven't received your code, try resending it now via this link.

I am having some problems with my emails at the moment actually (e.g. contact at overstream.net), this will be the last thing to be fixed.

Transferring Site Now

2008-03-12T02:33:00.000-07:00

OK, we are down for maintenance. This should take 1-6 hrs for the DNS propagation to the new server.

Maintenance Downtime Later Today

2008-03-11T10:03:00.000-07:00

OK, it looks like everything is ready for the move to the new server. The planned maintenance downtime is 1-6 hours. Again, you will get a different amount of maintenance time depending on where you are because DNS propagation is involved. The maintenance is planned to start on March 12 around 5pm Tokyo/Japan (8 am London/England, 4am East Coast/USA, 1am West Coast/USA, find your city).

Transferring Account

2008-03-10T08:05:00.000-07:00

Our IT guy Paul is transferring the account now. Tomorrow when I wake up I will try to verify that everything works, and then bring the site down and do the actual latest database transfer (so that no data is lost) and then DNS propagation. So, the downtime (2-6 hrs) will probably be sometime tomorrow. I will post more info when it becomes available.

New Server Being Set Up

2008-03-06T20:42:00.001-08:00

The new server is being currently set up for us by MyRackspace, our new hosting. This should be finished this weekend, and we can transition over early next week.

The current server is still suffering from the "crazy clock" problem, causing times on the site to appear very strange, and registration emails getting lost. However, I have opted to not take the previous hosting up on their proposal to move the existing server to another server of theirs because I simply do not trust them anymore. At least with the current server I know exactly what the problem is (even though it's a really bad problem). Also, I wanted to keep the downtime to a minimum, and this way we will only have a single transition.

Over the last month an improbably large number of unfortunate things have happened. I am really hoping that this curse will run out once we move to the new server. The good thing is that I now have an IT friend who is helping me with the server stuff, so I feel safer about all this.

Enough is Enough

2008-03-03T23:44:00.000-08:00

OK, guys, I can't take it anymore from our current hosting. They have screwed up so majorly both with the server configuration and the fact that the clock on the current server is screwed up, and they don't even feel sorry about it or try to go out of their way to fix it. I feel that they simply do not care. That is why we will be moving to a new hosting called "MyRackspace" - they are more expensive but their quality of techsupport is known far and wide. I am currently talking to them about the server specs, etc., so I imagine it will be a couple more days before the move.

I currently have the option of having the existing hosting move us to a different server (without the clock problem) - with about 3-4 hrs downtime. However, the current server works (for better or worse, with timestamps set in the future and the registration emails getting lost left and right because the time on them is totally wrong), and I simply do not trust the current hosting to configure the new server in such a way that there are no problems. So, unless forced into it I will try to avoid that move, so we can have a single period of downtime when moving to the new hosting.

On the current hosting's site it says at the top "Stop worrying about your server!". But all I've done ever since the new server was put into place was worry about how the site is running. So, I am sorry about all the problems (at least you can still create and watch overstreams!), and I sincerely hope we will be happier at the new hosting. I will post more about the move and the projected downtime when it becomes known.

Registration Emails Not Being Sent

2008-03-02T20:17:00.001-08:00

We are currenly experiencing a problem with the clock on our server being screwed up, which should be fixed by the hosting early this week.

One of the effects of this problem is that a lot of registration emails do not get through, or get put in the Spam folder - because they are sent out with a time stamp which is in the future. If you have registered, but did not get your registration email, please contact me at contact @ overstream.net, and I will gladly activate your account! Or you can just post a comment to this post requesting your account to be activated.

Server Clock Screwed Up

2008-03-01T00:51:00.000-08:00

It appears that the hosting has botched something when assembling the server, and the server clock is screwed up. What happens is that after the server is reboot, the clock is fine, but then it starts going successively faster and faster (covering 2 seconds in one, then 5 seconds in one, etc.). This screws up a lot of other things, causing me to reboot the server (which causes about 3 minutes of downtime, when none of the pages of the site load). The site generally appears to work even with the screwed up clock, but you might get some strange timestamps on your oversteams and comments until this is fixed.

I am (obviously) talking to the hosting about fixing this problem. Fixing it will result in some downtime, I will post later about how much and when it's going to happen. They also screwed up a number of other things in the server configuration as well... which overall makes me think that I made a mistake in the choice of hosting. Well, moving to another hosting means more downtime, and I am trying to avoid it, so unless the problems on the site become bad, I will try to stay with this hosting.

I feel like the curse is continuing...

Server Configuration Restored

2008-02-29T06:03:00.000-08:00

I have just noticed that an important server configuration file got erased during the move. I have just restored it. It's quite amazing that the server worked at all without it :) There were several small things that broke because of the absence of this configuration file, most noticeably you might have noticed that strange backslashes (\) started to appear whenever you would put a single quite (') or a double quote (") in your Overstream's Author comments - and maybe also in just comments, I am not sure. This should be fixed now.

Server Optimization

2008-02-27T22:58:00.000-08:00

Well, I can see at least everything is running - with some minor glitches, as reported by some users, but running! Over the next couple of weeks, I will try to figure out as much as possible how to optimize the server performance (especially with regards to the database), so I hope we can get past the glitches and the performance will be better!

Good Performance So Far

2008-02-27T15:29:00.000-08:00

So far the server has been responding pretty quickly, but of course we need to wait a couple of days and see what is happening with it. Hopefully, this whole move will be worth it ;)

Alright, I am finally going to go to sleep (8:30 am here ;)). I'll ask my friend to keep an eye on our server just in case.

Enjoy!

Liftoff!!!

2008-02-27T15:04:00.000-08:00

I think we have liftoff! At least I am seeing the site, and the videos seem to play on it :)

DNS Starting to Resolve

2008-02-27T14:35:00.001-08:00

Some users are already reporting that they are seeing the new site.

If you see "Project Damage Control", it is due to a slight misconfiguration on the part of the hosting - I am on the phone to them right now about it. This will be easy to fix.

Instructed Hosting to Switch DNS!

2008-02-27T13:13:00.001-08:00

OK, I figured there is only so much I can check without switching the DNS over. (For example, the logging into the site didn't work but I am pretty sure that was only because the domain of the site was not www.overstream.net).

So, I told the hosting to switch over the DNS! This means that once they have initiated the switch, we will just have to wait for the DNS to propagate - which could take 1-5 hrs depending on where you are. USA should be closer to 1 hr (because that's where the hosting is). In any case, keep checking the main site and at some point it should just come up.

Since you will probably see the site before I will, please post back here and tell me if something isn't working so that I am ready to fix it. Also, as I imagine there will be quite a stampede of users, this will be a real stress test for the young server ;) If anything doesn't work as expected, please tell me ASAP.

I will post when the hosting confirms that they have initiated the switch.

Thanks for sticking with me throughout this ordeal, and I hope we are almost there!

Currently Configuring Server...

2008-02-27T11:32:00.000-08:00

Well I was hoping things would just magically start working, but it's not quite so easy - I have to make sure that the web server, the PHP and the database are configured in the same way as on the old server. Currently trying to figure out the necessary server options... this is a first for me :)

Got the Configuration Email!

2008-02-27T10:35:00.001-08:00

Finally, I got the Welcome/Configuration email! In fact, I got it 35 minutes ago but it was stuck in my gmail Spam folder!

OK, now to figure out the status of the site. Once I verify that everything is working, I will instruct the hosting to switch over the DNS, and then we'll be in home stretch!

Files being copied RIGHT NOW

2008-02-27T07:20:00.000-08:00

After about 1.5 hrs on hold, I finally got through to the hosting. They are copying the files over right now from the old shared account. To copy the files they had to shut down the account, which explained a "Page Not Found" you might have briefly seen on the site there. In any case, the tech claimed that the actual copying process will not take more than an hour. We are getting closer!

Account Still Being Transferred!

2008-02-26T17:32:00.001-08:00

Eeek! Just got off the phone with the hosting techs, they said the account is STILL being transferred. Now I really don't see what should be taking so long to transfer the account, especially since I explained our situation previously and they agreed that they will try to do it as fast as possible.

In any case, the tech said that they were currently still working on moving the data over and promised that the account will definitely be transfered at most in the next 12 hours (which is really kind of a lot I think!). Well I am hoping their message about the transfer will come earlier than that. Guess it's a good thing I didn't stay up to wait for it to be transfered :)

Server Set Up, Account Being Transferred

2008-02-26T08:27:00.001-08:00

Just got this from the hosting: "Your server has been setup and your account is in the process of being moved." So it shouldn't be long before I can start verifying whether things are working or not!

By the way, I got a server with a RAID (that's the thing that makes sure that your server keeps going even if its hard-drive fails - it just switches over to another one), and also with Managed Data Backups - this is when the hosting backs up all the data on the server to a safe location in case something drastic happens to the server, like a tsunami, earthquake, or a terrorist/hacker attack. :) I know some of you will be happy about this last option :)

Dedicated Server Installation

2008-02-25T23:37:00.001-08:00

I have just called the dedicated hosting techs, and they told me that they got all the parts and the server will most likely be installed first thing in the morning (that's USA East Coast morning), so I should be getting the email with the info about it in around 6-8 hours. I also once again begged them to do anything possible to expedite the process, but apparently they don't do the actual installations during the night ;)

Once the server is installed, and the account transfered, I can start going through it to make sure that everything still works in the new place. I am hoping that this part will not take long, but as always we have to account for unforeseen difficulties. After I make sure that everything appears to work, I will give them the go-ahead to switch over the DNS, and after that it should only be 1-5 hrs for it to propagate (it was pretty fast last time, only took <2 hrs to propagate to Japan). So, things are moving right along.

Overstream Down For Maintenance

2008-02-25T10:05:00.000-08:00

The hosting techs have noticed that we were bringing the server down again, but this time they gave me a chance to put my own maintenance page up. I linked from it to this blog.

The techs have promised to expedite the process of the dedicated server installation as much as possible. As soon as any ETA (that's Estimated Time to Arrival :)) will be known, I will post it. Originally I was told that "Typical installation times are 24-36 hours, but in your case we can do it significantly faster". Let us hope so.

Oh, and in case you are wondering how come none of my questions about the dedicated hosting were answered for 3 days straight (before today), the explanation is: "I apologize for the delay. Your ticket was mistakenly flagged to the wrong department." It's a curse, I am telling you... :)

Stressing Out The Shared Hosting

2008-02-25T09:41:00.000-08:00

I can almost hear the shared hosting server creaking like a mast under a strong wind... I have a feeling they will probably suspend us again until we can make the transition to the dedicated server....

Unsuspended

2008-02-25T08:33:00.000-08:00

And we are currently back up! Hopefully the surge of eager of overstream users will not force them to suspend the account again until we can transition to the dedicated service...

I will post more about the transition to the dedicated service and the downtime associated with that later to give an advance warning so that you plan on not using overstream during that time (I know, it's hard ;))

Waiting to be Unsuspended

2008-02-25T08:08:00.001-08:00

Looks I like I have convinced the hosting to unsuspend the shared hosting account until I can move it to the dedicated server. They are taking a chance here - our account has screwed up the server enough times by now :)

Waiting to be unsuspended...

Dedicated Hosting Ordered

2008-02-25T07:28:00.001-08:00

I just got off the phone, having ordered a dedicated server hosting. It will take them some time to set it up, and then to transfer the existing account and then for me to go through it and make sure that everything is working. They said that this might take a maximum of 24-36 hours.

At the same time I am trying to get the shared hosting to unsuspend the account for the time being (while the dedicated server is setup), but I can understand their reluctance - our account has a knack for bringing down the whole shared server (which has 100+ other accounts on it :)).

In any case, the bottom line is that in at most two days we will definitely have a dedicated server. Let's hope that somehow I can convince them to bring up the shared hosting account in the meanwhile... keeping my fingers crossed!

Account Suspended Again

2008-02-25T06:41:00.000-08:00

Yep, I can see the hosting suspended the account again - due to all the traffic you guys have generated. And I was so hoping I would manage to calmly move to a dedicated server without any more downtime... :( But no, they wouldn't answer my emails about the dedicated server for 3 days ;)

Reaching Hosting Providers

2008-02-25T06:31:00.000-08:00

I contacted several hosting providers about their dedicated hosting plans, but so far have gotten no response (this after 2-3 days). I tried calling our current hosting, but could not reach them (they put me on hold for ~5 minutes several times, and then said they were experiencing unusually high call times). This feels like a curse or something ;) Well maybe it was just the weekend, I will try more during the week.

If you guys have dedicated hosting recommendations, do tell!!!

Dedicated Server Move

2008-02-21T23:13:00.000-08:00

It has come to the attention of our hosting that the server load due to overstream users is really getting waaaay too much for the shared hosting environment. So, we will be moving to a more expensive and higher-performing dedicated server account soon. I am looking at several dedicated server options right now.

Obviously, I will try to keep the downtime related to the move to a bare minimum. Also, I will try to schedule it when it's night everywhere in the world... :D I will post more info with exact information on when the move should happen. After this move we will have a faster and more reliable server, so that will be good :)

DNS Propagated (At Least for USA and Japan)

2008-02-20T09:53:00.000-08:00

O joy! I can see http://www.overstream.net working again! So, the DNS has propagated to Japan already - faster than I expected. It is also working in USA, some users are reporting. So, we are well on the way to recovery! The hosting promises they will keep an eye on the account to see if it's doing anything funny, but I can't imagine why it would - it worked just perfectly on this server all this time.

I will be finally going to sleep (it is deeeep night here in Tokyo), and I will try to sort out any remaining issues tomorrow. I guess I feel tired in the same way as Bilbo from There and Back Again: for some obscure reason we were moved to another server, it didn't work out, and now we are back home. :)

Domains Pointed To The Original Server

2008-02-20T09:03:00.000-08:00

Alright, the hosting technicians tell me that the move has been effected, and the domains have been pointed back to the original server. So, now we just wait for the DNS to propagate - some of you will see the site come up earlier than others, and I am crossing my fingers!

Since I am in Japan it will take me a bit longer to see the site come up.

DNS Propagation To Start Within The Hour

2008-02-20T08:28:00.000-08:00

I just got word from a technician at our hosting that he is currently in process of moving the account back to the previous server, and that the DNS propagagation should start "within the hour". This means that the server should be accessible to the people in North America in about 2 hours or a bit more, and the move should be completed world-wide in about 6 hours. I will keep posting the details.

Maintenance Message Is Up

2008-02-20T07:12:00.000-08:00

Finally, the hosting has managed to put the overstream maintenance message in place of the "suspended" message. We are proceeding with the account move back to the old server. I will report as soon as this move is made.

The last technician I have talked to has cited typical times of DNS propagation as 1-5 hours. The hosting is physically situated in USA in the East Coast, so for people in North America the DNS propagation time will be closer to the 1 hour mark.

Moving Back To Previous Server

2008-02-20T06:53:00.000-08:00

Well, it looks like we'll be moving back to the previous server, since the hosting simply can't figure out why our account is causing such problems on the new server. Oh well, who wanted that new server anyway! I am keeping my fingers crossed that the move passes with no problems.

The site will be inoperative during the move - it might take DNS ~2-3 hours or a bit more to propagate. In the meanwhile I am trying to get the hosting to change that ugly "this account is suspended message" which is so confusing to a reasonable maintenance message.

Hosting Suspended Our Account Again

2008-02-20T06:29:00.000-08:00

Overstream.net moved to the new server and is continuing to cause problems there for some reason (it should work, it worked just fine on the old server). This apparently is causing the hosting to "suspend" our account every now and then, so that all the visitors see when they go to overstream is an ugly "this account has been suspended" message.

I am trying to get through to them right now on the phone. Via email, we are also discussing the possibility of the account being moved back to the old server on which it has worked without problems. This move would entail shutting down the site for several hours while allowing the DNS to propagate, so that no user data is lost during this move.

And We Are Back Up!

2008-02-19T22:21:00.000-08:00

And right after this informative call I can see that overstream.net is back up!

Hosting Says Still Working On The Server

2008-02-19T22:18:00.000-08:00

Got through to the hosting once again, now they say that they have no accurate ETA on when they will finish the work on the server. The tech I got also couldn't tell me exactly what was being worked on either.

Hosting Says Server Back Up In 20 Minutes

2008-02-19T21:38:00.000-08:00

Just got through to the hosting, they said that they are "currently working on this particular server, it should be back up within 20 minutes". Just our luck that we get moved to a glitchy server... ;)

Server Down Again

2008-02-19T21:16:00.000-08:00

I can see overstream.net is down again. I have just called the hosting and got the message that "due to extraordinarily high volume of calls my call cannot be answered". So, it sounds like the hosting is having some major problem there.

Le Grand Content

2008-02-01T20:32:00.000-08:00

I simply love this film!

And now we can see it with spanish subtitles! :)

Japanese Tea Ceremony

2008-01-29T05:41:00.000-08:00