Welcome to MekTrix 4.0

Aka Shish’s Place of Stuff

Archive for the 'stats' Category

TravMap’s somewhat-late Birthday

I forget exactly when the project was started, but monitoring started a year ago. Thus I shall reminice:

Originally, a bunch of people in an IRC channel I hang out in decided to play Travian. As there were about 20 of us, keeping track was a pain, so I decided to make a map. This first version of the map was an SVG file drafted in Inkscape, edited by hand, and saved as a PNG image.

Pretty soon I got bored of updating by hand, and started looking for a way to automate things. Thankfully, the travian creators decided to be awesome and give out a dump of part of their database \o/ I put it into a local SQLite database so that it could be searched easily. Seeing as this was just a 5 minute hack that nobody other than myself would ever use or even see, I decided to use PHP to write the front end…

After a couple of weeks of adding features, I noticed that it could be useful to other people, so I posted about it on the forums. It ended up being quite popular, and I got several feature requests~ I also added support for other servers, and from there, to other languages. Having people offer to help with translation has been very motivational — It’s nice to know that people care about the project~

Talking about other languages, an interesting note on culture; when an American server is added, I get bug reports along the lines of “Type: Defect // Priority: Critical // The new server was added 3 hours ago, why is it not on travmap yet?!”. When a European server is added, the report is more like “Type: Enhancement // Priority: Minor // There is a new travian server at s2.travian.foo, please can you add it to your list? Thankyou :)”

Another few weeks later I had broken all my web serving records; I was serving tens of megabytes per day, and my server was melting — getting a P2 200MHz box with 64MB RAM to search through 200MB of database several times a second was painful. Switching to highly indexed MySQL made things better, but still bad. I ended up buying a load more RAM, ending up at 256MB. This was OK for a couple of months, until the database grew to be again too large to fit in memory… It got to the point where it was being queried faster than it could respond, so it got slower, and things stacked up exponentially — load average broke 60 within minutes of apache being started :-/

Eventually I gave up on self hosting, and moved across to dreamhost — thanks to the wonders of overselling, they had tons of spare processing power, memory, and bandwidth. Aside from datacenter problems, they’ve been pretty cool.

Since then, growth has been pretty steady. The database is now over 500MB, going over a hit a second at peak times (That’s quite a lot of traffic, nearing a gig a day now…). I also notice that dreamhost moved me to a different database server without telling me :|

Also I figured that with 10,000 visitors per day, there ought to be some way of making some money out of it — putting up a “please pay for my hosting” link earned me a grand total of $5 ($4.50 once paypal took their cut) from one guy — to this day, the only money I’ve made directly from programming ^_^ About a month ago I put some ads on the site to see how effective they were, and since then I’ve made enough to cover nearly half of that 256MB RAM I bought so long ago — another 3-4 months of the same earnings, and I’ll have got as much money out of the project as I’d put in XD

Overall, it’s been a fun year — I’ve learned lots about caching and optimisation, and how much PHP sucks (I *really* wish I’d written it in something else, I didn’t know it’d get so big…); I haven’t actually played travian for a few months now, I’m just sticking with the project because it’s enjoyable~

I forget how I was going to end my rambling, so I shall do so abruptly. End.

Posted March 15th, 2007 by Shish, in software, stats, tech, web

10k hits/day O_O

Looks like TravMap is very popular, pushing the site over the 10,000 hits per day mark :o

It’s had such an effect that it can be seen on the total-hits graph (it was released about new year):

Edit: comments off, since this post seems to attract more spam comments than all the others put together o_O

Posted February 19th, 2006 by Shish, in stats, tech, web

Joy of Stats

Some numbers, from site access logs:

Date range: 2004/09 to 2005/11

Size: 500MB

Time to parse with awstats: 5.5 hours

Files served: 2,237,523

Pages served to non-bots: 62,759

Front page hits: 4,553

Bandwidth used by visitors: ~35GB

Bandwidth used by bots: ~40GB

ED> And having left the config set to update all files, I note that only 3 hours was log parsing, with 2.5 being HTML output o_O

Posted December 6th, 2005 by Shish, in stats, tech

No more mrtg-cgi \o/

All stats are now generated by PHP calling RRDTool directly, which is much nicer~

Posted November 24th, 2005 by Shish, in pimp, software, stats, tech, web


The IRC archive activity graphs were converted from MRTG + RRD + CGIs to pure RRDTool; now they’re faster, prettier, and don’t require root access to change settings \o/

Posted November 21st, 2005 by Shish, in pimp, software, stats, tech, web

Linkage from More Random Places

My lame, old, probably copyright infringing wallpaper of Cloud in front of the ShinRa Building made in into the Italian Wikipedia o_O

Posted September 26th, 2005 by Shish, in stats, tech, web


Ho hum, at least this shows the automatic recovery systems work – I was back online and serving before I’d noticed PuTTY had disconnected~

I’m also back on the front page of hits for “Shish”, above MSDN \o/

Posted September 21st, 2005 by Shish, in problem, stats, tech, web


Third time lucky; after once barely getting an installation and once installing text mode only to get an infinite loop of database errors, Nagios works \o/ (login guest:guest)

I have it checking up on my services, so I get emailed eg when the IRC stats pages haven’t had their hourly update, or when various network services go offline.

Posted July 10th, 2005 by Shish, in pimp, software, stats, tech, web

A reorganisation, and yay Google \o/

Playing with google sitemaps, I found some interesting things:

* My avatar hosting is popularity 1.0

* Wallpapers are 0.05

* All my other pages are 0.0002 something

Also I realised that with /pics being one of the most popular bits of the site, it deserved a place in the main rather than /misc. Numerous other small changes have been made, but mod_rewrite should take care of redirecting everyone to where they want to go.

Posted June 14th, 2005 by Shish, in pimp, software, stats, tech, web

Skateparks & Jackass

I’ve suddenly started getting ~50 hits a day looking for “skatepark” and “jackass” images. I have no idea why. Someone please tell me! Email is good if you can’t be arsed to sign up to comment (which reminds me anonymous comments need setting up at some point…)

Posted March 31st, 2005 by Shish, in stats, tech, web