Welcome to MekTrix 4.0

Aka Shish’s Place of Stuff

Archive for the 'tech' Category

Shimmie Grows~

It’s getting people other than myself contributing to it, such that I’ve had to learn how to set up subversion for multiple users :O

A problem I noticed, and google didn’t seem to have the answer for — when the client says “svn: CHECKOUT of ‘[filename]’: 403 Forbidden ([host])”, and the apache error log says “Access denied: ‘[user]’ CHECKOUT”, note that authz paths mustn’t end in a slash, unless it’s the root. eg, this works:

[shimmie2:/]
* = r
shish = rw

[shimmie2:/trunk/contrib/link_image]
Artanis = rw

But this doesn’t:

[shimmie2:/]
* = r
shish = rw

[shimmie2:/trunk/contrib/link_image/]
Artanis = rw


Posted May 8th, 2007 by Shish, in problem, tech

TravMap’s somewhat-late Birthday

I forget exactly when the project was started, but monitoring started a year ago. Thus I shall reminice:

Originally, a bunch of people in an IRC channel I hang out in decided to play Travian. As there were about 20 of us, keeping track was a pain, so I decided to make a map. This first version of the map was an SVG file drafted in Inkscape, edited by hand, and saved as a PNG image.

Pretty soon I got bored of updating by hand, and started looking for a way to automate things. Thankfully, the travian creators decided to be awesome and give out a dump of part of their database \o/ I put it into a local SQLite database so that it could be searched easily. Seeing as this was just a 5 minute hack that nobody other than myself would ever use or even see, I decided to use PHP to write the front end…

After a couple of weeks of adding features, I noticed that it could be useful to other people, so I posted about it on the forums. It ended up being quite popular, and I got several feature requests~ I also added support for other servers, and from there, to other languages. Having people offer to help with translation has been very motivational — It’s nice to know that people care about the project~

Talking about other languages, an interesting note on culture; when an American server is added, I get bug reports along the lines of “Type: Defect // Priority: Critical // The new server was added 3 hours ago, why is it not on travmap yet?!”. When a European server is added, the report is more like “Type: Enhancement // Priority: Minor // There is a new travian server at s2.travian.foo, please can you add it to your list? Thankyou :)”

Another few weeks later I had broken all my web serving records; I was serving tens of megabytes per day, and my server was melting — getting a P2 200MHz box with 64MB RAM to search through 200MB of database several times a second was painful. Switching to highly indexed MySQL made things better, but still bad. I ended up buying a load more RAM, ending up at 256MB. This was OK for a couple of months, until the database grew to be again too large to fit in memory… It got to the point where it was being queried faster than it could respond, so it got slower, and things stacked up exponentially — load average broke 60 within minutes of apache being started :-/

Eventually I gave up on self hosting, and moved across to dreamhost — thanks to the wonders of overselling, they had tons of spare processing power, memory, and bandwidth. Aside from datacenter problems, they’ve been pretty cool.

Since then, growth has been pretty steady. The database is now over 500MB, going over a hit a second at peak times (That’s quite a lot of traffic, nearing a gig a day now…). I also notice that dreamhost moved me to a different database server without telling me :|

Also I figured that with 10,000 visitors per day, there ought to be some way of making some money out of it — putting up a “please pay for my hosting” link earned me a grand total of $5 ($4.50 once paypal took their cut) from one guy — to this day, the only money I’ve made directly from programming ^_^ About a month ago I put some ads on the site to see how effective they were, and since then I’ve made enough to cover nearly half of that 256MB RAM I bought so long ago — another 3-4 months of the same earnings, and I’ll have got as much money out of the project as I’d put in XD

Overall, it’s been a fun year — I’ve learned lots about caching and optimisation, and how much PHP sucks (I *really* wish I’d written it in something else, I didn’t know it’d get so big…); I haven’t actually played travian for a few months now, I’m just sticking with the project because it’s enjoyable~

I forget how I was going to end my rambling, so I shall do so abruptly. End.


Posted March 15th, 2007 by Shish, in software, stats, tech, web

On Servers

Chai (router) and Aru (server) are being turned off for the first time in 4 months; chai is retiring for now, aru is becoming a router + xen server. It’s pretty much the same setup as I’ve done in school — one upstream network, two downstream; serving DHCP on the two downstreams using dnsmasq. VPSes are bridged to one or more networks depending on who needs access to them.

The school’s moodle install has been moved to a VPS at context shift, since dreamhost were complaining that we were using too much CPU time. That seems to have gone mostly OK, except for me not knowing how to configure Apache + PHP to deal with more load than the server can handle (Lowering MaxClients seems to have solved the swappy death problem — now if the server gets more requests than it can handle it queues them, rather than trying to do everything at once)

Update: FastCGI’ed PHP seems to ignore MaxClients. Sigh.


Posted January 4th, 2007 by Shish, in pimp, reallife, site, tech

Tech support at UKC

Blogging of my first days at uni has been delayed by problems connecting to the network… And the resulting conversation with tech support:

From: shish
To: tech support

Your SBS signup form for linux users [1] has typos[2] that make it
impossible to submit; the resulting page is a back trace of a
nullpointerexception[3]. I guessed the URL of the windows version of the
form and filled that one in instead, and it works fine (it even sent me
the correct download, CSA.sh). The fix is to go through the form and
give the elements their proper names (eg, change "username0" to
"username")

[1] http://reg-kent.org:8080/registration/register-linux.html

[2] It looks like someone copy & pasted the form in a WYSIWYG editor --
all the relevant elements had zeroes after them, which is a standard
way of keeping unique element IDs after copy & pasting...

[3] I would guess somebody did something like form["username"].value,
without first making sure that the "username" element was POSTed :-/

A rather simple problem, and I even took the time to work out why things were broken, and explain how to fix them in easy to understand terms. The result?

From: Tech support
To: Shish

Thanks for your e-mail.

You need to have Java turned on in order to see the page correctly and submit it.

Gah, a boilerplate response, and an incorrect one at that, on a variety of levels — there is no java on the page, so enabling it won’t make any difference. I would assume he meant javascript (something totally different, the name is coincidence), but that is inappropriate too, since the problem is nothing to do with the script (which works fine in my already javascript enabled browser).

So I reply again, in even simpler terms, and explain why the previous response was wrong:

No I don't; there is no java on that page. There is javascript (which
is a different thing altogether), and I do have that enabled. I know my
browser works fine since I can use the version of the page designed for
windows users.

To repeat: The problem is that the linux version of the page has errors
in its code that cause the values entered to be given the wrong names.
For example, the linux username box is called "username0", and doesn't
work, whereas the windows and mac pages have called the box "username"
and they work fine. If I download the registration page and change the
name of the box myself, that works too.

It's a very simple problem with a very simple solution; you could
probably fix it in less time than it would take to send another
useless boilerplate message :P

A response?

Thanks for your e-mail.

You need to have Java turned on in order to see the page correctly and submit it.

Argh.


Posted September 18th, 2006 by Shish, in problem, rage, reallife, school, tech

Hurrah for non-sucky kernels

I have 20 autoconf / compiler processes running, my music plays without a jitter, and my UI is as responsive as when it has the whole CPU to itself — I never realised how apparent the new linux scheduler was until I saw that it was giving a single interactive process more time than all the batch processes combined~

Compare windows, where a single cpu / memory hogging program can render the whole computer unusable…

ED: Locking comments, since there are a couple of spams per day and no useful comments…


Posted July 4th, 2006 by Shish, in pimp, software, tech

Oh dear lord…

Sites like this make me regret my free hotlinking policy… The furryism I can forgive*, the stupidity I can stand*, but the web design is so eye-blisteringly ugly I’m ashamed to be associated with it x_x

* so long as they keep it to themselves…


Posted July 4th, 2006 by Shish, in web

Eating PEARs~

For anyone with the same problems I had…

PEAR from debian stable is too old and won’t upgrade: download 1.3.5 manually, run “pear upgrade PEAR-1.3.5.tgz”; download the most recent version and run “pear upgrade PEAR-1.4.8.tgz” (in my case). You should then be able to let PEAR upgrade itself from then on.

MySQL extension is installed, but you still get the warning warning: pear/MDB2_Driver_mysql requires PHP extension "mysql": Make sure “module=mysql.so” is uncommented in /etc/php4/cli/php.ini.


Posted June 22nd, 2006 by Shish, in problem, tech

Expected Downtime

Aru, the main home server and shishnet development & testing site, now has a 300GB disk \o/

Uptime was 3.5 months, since some new RAM was installed~


Posted May 21st, 2006 by Shish, in tech

Unexpected Downtime!

After 8 hours of being under full load, chai (my linux router) panicked, taking the network offline for 3 hours. There’s nothing *obviosuly* wrong with the hardware, so I assume it a linux problem. I would have investigated properly, but the stack trace only showed the addresses of functions, not names…

Seeing as one error per 4 years is unacceptably high (from debian stable, of all distros!), I’ve since started looking into the BSDs; particularly m0n0wall.


Posted May 21st, 2006 by Shish, in pimp, problem, software, tech

dreamhost /o/

Again with the lateness, I’ve finally gone and bought some external hosting — the new server is at least 10x better than my self-hosted one in all respects, over 1000x in others. It’s shared with a hundred odd other people, but so long as nobody wants all the power at once, it’s hardly noticable~

Not only do they not suck, which puts them above most web hosts, they’re also actively good — their web based control panel actually makes a lot of things easier than going to the shell and doing it all by hand. Things that other hosts don’t have at all (Subversion, Jabber, raw log access, unlimited subdomains, multiple shell / mail / ftp users, per-site bandwidth throttling, etc), they make no more than a couple of clicks away \o/

They also installed rrdtool at my whim (I do make extensive use of it, but they don’t know that…), using what appears to be the ideal management system (Custom APT repository + bulk server-management tools). They also cite a couple of moral reasons amongst the many technical ones for being a linux-only host, and will match any customer’s donations to their charities of the month :O

And last but not least, they have a sense of humour which matches mine <3


Posted April 26th, 2006 by Shish, in pimp, reallife, site, tech, web