Welcome to MekTrix 4.0

I forget exactly when the project was started, but monitoring started a year ago. Thus I shall reminice:

Originally, a bunch of people in an IRC channel I hang out in decided to play Travian. As there were about 20 of us, keeping track was a pain, so I decided to make a map. This first version of the map was an SVG file drafted in Inkscape, edited by hand, and saved as a PNG image.

Pretty soon I got bored of updating by hand, and started looking for a way to automate things. Thankfully, the travian creators decided to be awesome and give out a dump of part of their database \o/ I put it into a local SQLite database so that it could be searched easily. Seeing as this was just a 5 minute hack that nobody other than myself would ever use or even see, I decided to use PHP to write the front end…

After a couple of weeks of adding features, I noticed that it could be useful to other people, so I posted about it on the forums. It ended up being quite popular, and I got several feature requests~ I also added support for other servers, and from there, to other languages. Having people offer to help with translation has been very motivational — It’s nice to know that people care about the project~

Talking about other languages, an interesting note on culture; when an American server is added, I get bug reports along the lines of “Type: Defect // Priority: Critical // The new server was added 3 hours ago, why is it not on travmap yet?!”. When a European server is added, the report is more like “Type: Enhancement // Priority: Minor // There is a new travian server at s2.travian.foo, please can you add it to your list? Thankyou :)”

Another few weeks later I had broken all my web serving records; I was serving tens of megabytes per day, and my server was melting — getting a P2 200MHz box with 64MB RAM to search through 200MB of database several times a second was painful. Switching to highly indexed MySQL made things better, but still bad. I ended up buying a load more RAM, ending up at 256MB. This was OK for a couple of months, until the database grew to be again too large to fit in memory… It got to the point where it was being queried faster than it could respond, so it got slower, and things stacked up exponentially — load average broke 60 within minutes of apache being started :-/

Eventually I gave up on self hosting, and moved across to dreamhost — thanks to the wonders of overselling, they had tons of spare processing power, memory, and bandwidth. Aside from datacenter problems, they’ve been pretty cool.

Since then, growth has been pretty steady. The database is now over 500MB, going over a hit a second at peak times (That’s quite a lot of traffic, nearing a gig a day now…). I also notice that dreamhost moved me to a different database server without telling me :|

Also I figured that with 10,000 visitors per day, there ought to be some way of making some money out of it — putting up a “please pay for my hosting” link earned me a grand total of $5 ($4.50 once paypal took their cut) from one guy — to this day, the only money I’ve made directly from programming ^_^ About a month ago I put some ads on the site to see how effective they were, and since then I’ve made enough to cover nearly half of that 256MB RAM I bought so long ago — another 3-4 months of the same earnings, and I’ll have got as much money out of the project as I’d put in XD

Overall, it’s been a fun year — I’ve learned lots about caching and optimisation, and how much PHP sucks (I *really* wish I’d written it in something else, I didn’t know it’d get so big…); I haven’t actually played travian for a few months now, I’m just sticking with the project because it’s enjoyable~

I forget how I was going to end my rambling, so I shall do so abruptly. End.

This entry was posted on Thursday, March 15th, 2007 at 7:58 am and is filed under software, stats, tech, web. You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site.

1. Marco Says:
   March 22nd, 2007 at 3:18 am

   Hi!
   Could you tell me, please, how to access the public dump of a travian server database? where I can find that dump ?

   Thank you ;)

2. lance Says:
   March 22nd, 2007 at 7:55 am

   Is there anything published on how to access the Travian Data? I know of the map.sql file but is there anything else?

3. Shish Says:
   March 22nd, 2007 at 8:37 am

   The only data I know about is the map.sql file; if there is more, I’d like to be told :)

   map.sql is available as “http://[server]/map.sql”, eg “http://s3.travian.com/map.sql”, and the schema is:

   CREATE TABLE [whatever] (
   	lochash MEDIUMINT UNSIGNED PRIMARY KEY NOT NULL,
   	x SMALLINT NOT NULL,
   	y SMALLINT NOT NULL,
   	race TINYINT NOT NULL,
   	town_id MEDIUMINT UNSIGNED NOT NULL,
   	town_name  CHAR(20) NOT NULL,
   	owner_id MEDIUMINT UNSIGNED NOT NULL,
   	owner_name CHAR(16) NOT NULL,
   	guild_id MEDIUMINT UNSIGNED NOT NULL,
   	guild_name CHAR(8) NOT NULL,
   	population MEDIUMINT NOT NULL,
   	INDEX(town_name), INDEX(owner_name), INDEX(guild_name),
   	INDEX(owner_id), INDEX(guild_id), INDEX(x), INDEX(y),
   	INDEX(race), INDEX(population)
   );

4. lance Says:
   March 22nd, 2007 at 11:32 am

   Thanks for the layout info. I wish they’d give a list of unoccupied territories and the counts of resources in each “square” – that would be pretty cool.

5. Johannes Says:
   April 8th, 2007 at 7:59 am

   You rock. Thanks. (Btw, I’m european ;) in you face americans)

6. Genjio Says:
   April 11th, 2007 at 7:55 pm

   Thanks for being such an awesome person. thats about all i have to say.

7. jyri Says:
   April 20th, 2007 at 10:14 am

   So what language do you think would have been best for the job then if PHP sucks that much?

8. Shish Says:
   April 20th, 2007 at 3:46 pm

   Python, which is what I’m re-writing it in right now :) Specifically with mod_python, because regular CGI python has a lot of startup overhead for each request~

9. mars Says:
   April 28th, 2007 at 2:13 am

   Nice work Shish!

   Can you post some info about how you grab the sql into the database using php please?

   many thanks for the map and the info.

10. Shish Says:
    April 28th, 2007 at 1:40 pm

    I don’t do database imports with PHP, I have a pile of perl + bash + sed + mysql command line :P

11. MM23 Says:
    June 9th, 2007 at 12:45 am

    You’re going to hate me for saying this, but:

    Ruby on rails.

12. Shish Says:
    June 10th, 2007 at 11:01 am

    … why? RoR seems overkill; it makes the generic parts of webapp writing easy, but travmap is 99% non-generic. I don’t think that writing that 1% of code twice as quickly is worth the overhead of using any app framework~

    Having redone 75% of it in python (and done it properly this time), I’m realising that there’s such a high ratio of number crunching to webapp, it might be worth doing the whole thing in C :3

13. aeinstein Says:
    June 30th, 2007 at 6:42 am

    So where is this place? Theres no link to to page!

14. Seiti Says:
    October 2nd, 2007 at 10:02 am

    Thanks! Now I’ll play a little with mysql…

15. Skalbagge Says:
    December 2nd, 2007 at 10:18 pm

    Thanks for the table schema, Shish!