I was originally planning to write about using the C APIs of PostgreSQL, MySQL and SQLite, but as I don't have time for that today,I'm going to talk about something I've been doing the past couple of days: Migrating from SQLite to PostgreSQL for both my forum and my MUD.

First you might be wondering why I'm using the less supported PostgreSQL instead of MySQL. Frankly I will not enter that hornet's nest of a discussion here. Flame wars rage on the internet on that subject, and it's just as silly as "my programming language is better than yours". You use the tools that fits your purposes. Simple as that. In the following I'll talk about migrating both the forum and the mud in a single discussion, because they're both SQLite -> PostgreSQL anyway.

The first thing I did was to dump the entire databases into a text file of SQL statements. All SQL engines I've seen have tools for doing this. Technically you can simply read that same file into another database. However, it's rarely that simple. PostgreSQL has different syntax that SQLite for such things as making an autoincrement field:

SQLite:
CREATE TABLE test_table (
        a INTEGER PRIMARY KEY,
        b INTEGER
);

PostgreSQL:
CREATE TABLE test_table (
        a SERIAL,
        b INT
);

So obviously I had to change a few schemas. Using these autoincrement fields is also different:

SQLite:
INSERT INTO test_table VALUES ( NULL, 123 );

PostgreSQL:
INSERT INTO test_table VALUES ( DEFAULT, 123 );

So this forced me to change a few SQL statements scattered around the code as well. But it doesn't stop there. Case insensitive matching in queries also differ:

SQLite:
DELETE FROM players WHERE name LIKE 'caius';

PostgreSQL:
DELETE FROM players WHERE name ~* 'caius';

This is because the LIKE operator has slightly different meaning in the two engines. Again I had to change quite a few SQL statements in my code. At this point importing the data into the PostgreSQL database, and reading from it worked properly. However, I soon encountered another problem. Postgres refused to insert new rows into tables with autoincrement fields! Arg! The reason is that the engine keeps a separate sequence table to keep track of the index. So I had to update this counter manually. If we assume the top id in the above table is 14, we must send the following query to the database:

PostgreSQL:
ALTER SEQUENCE test_table_id_seq RESTART WITH 15;

So it's in the form "table name" + "field name" + "seq". Now, FINALLY, everything ran smoothly.

Now, you may wonder why the database engines doesn't do these things similarly? The reason is that they implement extra features that aren't a part of the SQL standard, and they each do it their own way. This is why migrating SQL data can be a pain, especially if you use non-standard features.

That's it for today. Cheers. 

After I started working with C++ I found that the compile times increased drastically. Especially using lots of templates, as in the case of the Standard Template Library, compiles incredibly slow. My codebase used to compile in about 30 seconds, but after rewriting substantial portions to C++ it increased to almost three minutes. As we all know, drastic times calls for drastic measures. So I'm going to delve into some of the solutions I've started using. You can benefit from these even if you're using C, as long as you're using the GNU Compiler Collection (GCC).

ccache
ccache, or compilercache as it originally was called, is a program that stores all the information about your compiles, so that when you recompile your code it can pull from the cache the files that are unchanged since last compile. This cache is stored in a directory called .ccache.

Using it is amazingly simple. You just run gcc/g++ through it. So if you compile a file called myfile.c you simply type: ccache gcc myfile.c. The first time you compile a piece of code the cache is built, so obviously this may be slightly slower than a normal compile, but not by much. You can also use it in a Makefile by prefixing gcc or g++ with ccache. In my example, the three minutes are reduced to 15 seconds! No, I'm not bluffing.

Keep in mind that it can only pull unchanged files from the cache, so if you change a header file, all files that includes it will be recompiled normally.

If your Linux/Unix distro doesn't come with ccache, you can get it here: http://ccache.samba.org/

distcc
Now we're getting a little more advanced. distcc is a distributed compiling system. For this you need at least two computers, connected on a high speed line. After installing and configuring this software, you can distribute your compiling jobs across multiple computers. And it's fairly simple to set up too.

You can think of this as a master/slave server setup. One of the computers will control the entire job, while the others simply listen, receive, compile and send back the results. After setting up the configuration you start the daemon called distccd which will then run in the background, listening for requests for compiling. The other program you will use is called distcc and is used just as ccache. Ie, you prefix gcc or g++ with it. At this point, if you simply type make it will only compile locally. However, make has an option called -j, which will start multiple compiling processes at once. So if you type 'make -j4' it will start four compiling processes at once. This is snatched up by distcc, and it distributes the job among the computers it's configured to know about. Using my desktop and laptop together my compile time is almost cut in half.

distcc can use two protocols: http and ssh. ssh is safer, but a little slower because it must encrypt the data first. So if you're compiling in your private LAN you should prefer http. It's also possible to compile over the internet, but then you will need fast connections between all involved computers to avoid bottlenecks.

distcc works with C, C++, Objective C and Objective C++. Oh, and I want to kill one myth right away: distcc is not a compiler, it's a front end for GCC. The final thing I want to say on distcc is concerning compiler versions. Although I haven't tested this myself, it's reported to work even if the computers involved have different versions of GCC, as long as they're binary compatible. I expect this is more often the case with C than with C++.

For more information on setting up the system, see here: http://distcc.samba.org/ 

The big question
Can ccache and distcc be combined? Yes! In fact, they cooperate incredibly well. It is recommended to run ccache first, like this: ccache distcc gcc. According to the distcc FAQ ccache will work better this way.

So, with these tools in your arsenal you can leave slow compiling in the dust. Happy coding, and I'll hopefully be back soon with another blog. 

Over the years I've used three different types of MUD host. These are; Free Shared Hosting, Commercial Shared Hosting, and Virtual Private Server. In this blog entry I'm going to talk a little about all these models.

Free Shared Hosting
There are those who will offer to host your mud on their server for no charge at all. Over the years there have been many of these, some disappearing after a few months, others staying around for years. The reason I call it Shared hosting is because you generally live side by side with a number of other mud owners. Now, there is essentially just one positive thing with a Free Shared Host, but it's a big one: It's completely free. How can you beat that? Well, actually you can, depending on your requirements.

The disadvantages are numerous. First off you can't expect any support. You may get support, even good support, but you can't depend on this. The second disadvantage is that you can't expect the 99.997% uptime that the real pros can offer. The server can often be unavailable for shorter or longer periods. Thirdly it's unlikely that the host will install all the software you want, or configure everything to your taste. For smaller applications and libraries you can of course install them in your home directory. But what if the host uses MySQL, but you really need PostgreSQL? What if the host doesn't have a database server at all? Then you're basically screwed, unless the administrator is unusually flexible. The final disadvantage that comes to mind is that you're not safe from sudden and permanent shutdown. Certainly, a commercial host can also shutdown, but in the case of a free host there's no guarantee that you will get a warning, or indeed be able you retrieve your code.

Note that some free hosts are actually just as good, if not better, than commercial ones. And others are more or less run from a 13-year old's bedroom over a DSL connection. More on selecting a free host below.

When to use a Free Shared Host? Provided you can get an account with one of the better ones, a free host is great during the early stage of your MUD. As you work on code and content, before opening the doors, a free host can be a great alternative. But when you're getting close to accepting players you should seriously consider one of the alternatives below.

How to select a Free Shared Host? Although I'm sure some will be offended by this, I have to say that the single most important thing is: How long have the host been running its service? If it has done this for a few years, it's a good chance they'll stick around a little longer. If it opened a few weeks ago, then use great caution. This fresh host can be great, but you can also risk that it goes down in a month, that the administrator is very inexperienced in running a server, and/or that the server haven't been fully configured yet. Next try finding information about the server(s). Is it hosted at home, like this: http://caius.totj.net/hosting.jpg ? Is it in a proper datacenter? Somewhere in between? And last, but certainly not least: Talk to people who are, or have been, using the host.

To sum up: Use a free host while you're still closed for players, and spend some time finding the right one.

Commercial Shared Hosting
The biggest advantage over a free host is the uptime, reliability and support you can expect. Some of the same disadvantages still applies. In fact, I've sometimes even found it harder to get commercial hosts to install new software. They tend to be conservative in this area because they are (with good reason) concerned about the integrity of the system. That being said, don't fool yourself. You can in most cases live with older versions of software. You probably don't really need GCC 4.2.x. Version 3.4 is fine in almost all cases. Similarly you probably don't need the latest versions of PHP, MySQL, etc. The problem is, again, if you need software that is either too large to install in your home directory or that needs to run in the background as a daemon/server process.

When to use Commercial Shared Hosting? You should leave the free host behind when you start accepting players, or preferably some time before that. The advantage of this hosting scheme compared to VPS (see below) is that you don't have to worry about administering the server. Let the pros handle it while you concentrate on developing your MUD.

How to select a Commercial Shared Host? Naturally you'll be interested in the price. Depending on the host, and depending on the resources you need, price can vary from $2/mo to $20 or more. Pay attention to things like disk space, RAM, and CPU quotas. As an example, my C++ mud takes 55MB disk space when compiled with full debug symbols (-g3 option in GCC), and it's not even bloated. C++ object code takes a lot of space. So having two instances of the mud (player port and development port), a code repository and a web site can easily require about 150MB or so. If you have to install additional software in your home directory we're looking at 200MB+. I recommend going for at least 300MB. On the other hand you may also choose to start small and upgrade your plan as you need it. Also, if you develop your mud separetely from the player port (and you really should), you will want to have at least two ports assigned to your account. Some hosts will give you this automatically, others will require you to upgrade your plan. So to sum up: Compare the various hosts carefully, and make sure you know your requirements in advance.

Virtual Private Server (VPS)
If you're handy with a Linux shell you may want to consider a VPS. A VPS is basically a virtual computer, so you will have root access and in most respects be the administrator of your own server. The advantages are obvious: Install exactly the software you want, create additional user accounts for your other coders (if applicable), unlimited number of emails, tweak all settings to your needs, use as many, and which, ports as you want, etc. You're Master of the House!

However, with great power comes great responsibility (sic). Unless you're willing to pay for support you have to deal with all the headaches of running a server. A lot of the services you need can be tricky to configure just right, and when something goes wrong you can only point your finger at yourself. Another potential problem is that, unless you've got a thick wallet, you may find that you run out of memory quickly. The cheapest plans comes with anything from 64 to 256MB RAM, which will require you to get a little creative when running a few server processes. Unlike a real dedicated server you usually can't set up your own swap file beyond what your provider gives you. On the other hand, they also tend to come with great processing power and plenty of disk space. This hosting alternative will require you to spend much time on the server in addition to developing your MUD. You should also be aware that you can't expect to get exactly the Linux distro you want. The provider usually gives you a handful of disk images to choose from. On the other hand it usually takes less than a minute to install a different distro, so you can try them all.

When to use VPS? When you want (almost) full control over everything.

How to select a VPS provider? As with anything else the prices vary a lot, and they're not always a measure of what you get. They can range from $6/mo to $100/mo or more. Personally I found a good deal with 128MB guaranteed RAM (256MB burstable), 10GB disk space, powerful processors, 100GB bandwitdh/mo, running CentOS 5 (more or less same as Red Hat Enterprise Linux 5). For this I pay $8.50/mo, which I think is a very good price. The only problem is the limited memory (I'll talk about that in a later blog). There are a lot of providers to choose from, so I advice spending time before picking one. Most of them have special deals every now and then, so keep your eyes open.

Conclusion
My recommendation, then, is starting out with a free host, then move on as your project outgrows it. When choosing between Commercial Shared Hosting and Virtual Private Server your main concern should be whether you're ready to be a system administrator or not. Don't assume that a shared host is cheaper, because it's not always true.

Good luck with your MUD. I hope you found some of these points useful. 

Been using Apache2 for webserver since moving to a dedicated server (or rather Virtual Private Server to be precise). While Apache2 is a good webserver, it's on the heavy side when it comes to memory. My VPS has limited RAM, so this has caused some annoyance every now and then. So I decided to look for alternatives. My choice eventually fell on Lighttpd, since it's supposed to be a decent webserver with low memory footprint.

Now changing the webserver can be a hassle, and this transition was no exception. The main issue I had was with PHP. I had to recompile PHP with a couple of extra options. Everything was fine until I fired up the forum which uses SQLite as a database engine. For some odd reason SQLite wasn't compiled into PHP even though I chose it in the configuration. After a lot of trail/error, and even more swearing, I eventually was able to load SQLite as a dynamic/shared module. Finally!

Now that (almost) everything works as it used to I'm pretty satisfied with going for Lighttpd. Indeed it requires only about half the memory as Apache2, with no real performance penalty. Apache2, being a heavy-duty webserver, is really overkill for a website dedicated to a MUD. We're unlikely to ever have 500+ simultaneous connections 24/7. Lighttpd is meant for sites with a light load, and we certainly fit into that category

The only issue I have with the new webserver is its poor webdav support. I've been using this to access the Subversion repository. However the svn+ssh protocol is fine. I now have to enter my password every time I access the repository, but this is a small price to pay for freeing up valuable RAM.

So, in short, I can recommend Lighttpd for anyone needing a leaner alternative to Apache. The address is http://lighttpd.net/ if you want to check it out. It comes with a good selection of modules, so it probably fulfills most normal scenarios. Took me about an hour or two to get it fully working as a replacement to Apache. Thumbs up!