Welcome to MekTrix 4.0

Because the world doesn’t have enough anecdotal evidence already:

On my gaming / media PC (custom built):

Graphics card, NVidia 8600 GT: ubuntu detects my monitor’s native resolution, runs ok using the standard drivers, and pops up a message asking if it should download the accelerated drivers automatically; windows runs at 640×480 until you install accelerated drivers from CD or google.

GPU accelerated h264 decoding: Linux yes, windows no.

Webcam, labtec thing: linux works out of the box; windows requires drivers from CD, and then the drivers crash whenever they’re used (and sometimes when they aren’t)

Extra USB ports: Linux PCMCIA worked in a fraction of a second (it was running faster than I could look at the “connected hardware” screen), windows PCI worked fine after 5 minutes of “you have new hardware!” “It is a PCI card!” “The PCI card has USB ports!” “There are 4 ports!” “Each port is USB!” “The USB version is 2.0!”…

TV capture card, Hauppage PCI thing: Ubuntu works out of the box, windows requires googling for drivers, and those drivers install crappy utility software.

Phone (Sony Ericsson k750i) as a modem: Ubuntu detects that the phone can be used as a modem, and pops up a message asking for phone service login details; windows detects the phone can be used as a modem, haven’t yet figured out how to use it.

Phone as USB storage: Linux works out of the box, windows requires googling for drivers.

Phone as bluetooth remote control for media player: Ubuntu works out of the box, windows works out of the box (No problems, no glitches, no googling for drivers; well done microsoft!)

And for the other PC (IBM thinkcentre):

Graphics (onboard intel 865): Ubuntu 9.04 broke 3D acceleration, all other distros and other versions of ubuntu work out of the box; windows works after googling for drivers

Sound (onboard intel something): Linux works out of the box, including features like disabling onboard speakers when external speakers are plugged in, and disabling external speakers when headphones are plugged in; windows works without advanced features after googling for drivers (and these particular drivers are a bitch to find. I suspect that I might not even have exactly the right ones, hence the feature lack :-/)

Network (onboard intel thing): Linux works out of the box, windows works after googling for drivers… yes. For those who haven’t spotted it, the /network card/ requires downloading drivers using /the network/ :-P

Network (atheros based wifi): Linux works out of the box, windows requires separate drivers

There is a “media” share, which anybody can read, and the media user can write to. I’ve connected to this share, but can’t write to it. My first thought is that I must be connected anonymously, so I right click -> disconnect… and nothing happens. Right click -> disconnect again, “this drive is not connected”. Hmm, explorer bug? Well, if it’s not connected, then I shall attempt to connect. So I go to connect to network drive, click “connect with different login details”, enter details for the media user, click connect, and … “this drive is already connected”, and then a second error message immediately follows “you cannot connect to the same share with two different accounts” (What sort of stupid limitiation is that? :-|). Well, at least it confirms that I am logging in with two different accounts, and thus used to be anonymous and am now media. Or does it? I reboot to get rid of the phantom connection, and try to make a new connection with the media login. It works, but isn’t writable! Giving up on the client side, I check the server logs: “connect to service media initially as user nobody”. Gaaaah, goddamn it windows, I explicitly said “connect as the media user”!

Gaaaah.

</rant>

The guide to Sarge I found here very nearly worked, but with some problems.

Quick summary of changes:

• where libpam-ldap and libnss-ldap suggest an LDAP URI of ldapi:///, ldapi://127.0.0.1/ doesn’t work — ldapi:// seems to be for unix sockets only. ldap:// or ldaps:// should work (I’ve tested ldap:// so far, as it was communicating with localhost)
• The /etc/pam.d stuff was screwy — where that guide suggests replacing your pam_unix.so lines with completely different ones, I suggest replacing them with copies of themselves but “required” replaced with “sufficient”. Screwing with my existing pam_unix.so config broke the ability to change local user’s passwords.
• In common-password, the “use_first_pass use_authtok md5″ after pam_ldap.so gave me a “passwd: Authentication information cannot be recovered” error when trying to change the password. Removing them (so that the line simply reads “password sufficient pam_ldap.so”) works.
• To clarify; the general idea of the common-* files should be “unix auth is sufficient, if that fails then ldap auth is sufficient, if that files then denying access is required”. I suspect that “unix auth is sufficient, if that fails then ldap auth is required” would be the same, but IMHO that’s uglier and more prone to breaking if people want more layers later.

Sorting through some of my old code, I found a link to one of my old websites; the site itself is long gone, though googling it did bring back memories; days of hunting down “host 1mb for free”, “get a free subdomain”, etc sites; hosting things on my 56k modem. Also, playing SMAUG and having parents yell at me because I’d been using the internet for a whole hour, and it cost a penny a minute! Then downloading 200MB of software development stuff in 1.4MB chunks that took 3 hours each…

Screenshot! Mektrix v2, in Mozilla (it was modern at the time), running on one of my first linux setups XD

Coniston’s network card died, and I was the nearest employee; thus, my turn to run up to the datacenter and replace it /o/

This would be easier if I had parts, or the tools to attach them -_-

Wilkinsons do screwdrivers though, which is useful \o/ Though they are cheap and bend when you attempt to unscrew things with them /o\ Even less usefully, it seems that wired networks are dying off — All the high street stores had shelves full of wireless cards, but at best 2 or 3 ethernet models, and those were 10/100 v_v

After trying a few shops and being pointed back and forth, I ended up at yoyotech, which seems to be a place of at least some clue; the sales dude (Brian, apparently) asked if I was using linux, and googled to see if the model of card was sufficiently supported~

Having bought it (and a second of another brand, just in case), I was off to the datacenter /o/ The trip was relatively short and uneventful, aside from a youth in a shopping center throwing a big enough tantrum for the police to take note :-/

The DC itself was relatively normal, with coolers and cables and such; the part exchange went well once I’d figured out how to get the side panels off, and what to do with the switch bolted to the side o_O

On the way home, the train was declared unfit for human transport, so we all had to get off and wait for the next, which was much more crowded :( But then I was home \o/

I don’t know if I’ve ranted on this last time, but it happened again with another site, so I shall rant again. For context, I’m the technical admin of a certain large site, which I won’t even name due to NSFWness — I’m just mentioning it here so that visitors from that site know who I am :) A brief timeline of events:

• 2 days ago, I move the database to a separate server, as MySQL grinds to a halt when the database is too big to fit in RAM.
• At around 11pm yesterday, the site suddenly dies, printing “500 Internal Server Error” in response to all requests
• Restarting the web server fixes it, but the old PHP processes don’t die, and the new ones lock up pretty quickly too
• PHP isn’t actually printing any error messages, it’s just locking up; LigHTTPD only prints “All PHP processes are busy, try again later”; MySQL logs show no error messages.
• It seems that the site is fine for as long as nobody tries to post new content.
• I try stopping and restarting the database, it says “Can’t stop database”
• I finally realise what happened — the disk with the database tables filled up, then MySQL crashed and died horribly, corrupting chunks of the tables. Then when PHP asks the database what’s up, and the database says “error”, it doesn’t print “there’s a database error”, it just dies horribly and locks everything up, causing ISE’s and using up all the ram until everything around it dies too.

In contrast, my preferred database server, Postgres, ran fine throughout the incident — read-only requests were served as normal, and if anything tried to add data it would be told “the disk is full”; no crashing, no corrupt data, no problem. Also, it seems to scale better, currently dealing with a database of 8 * the size of RAM quite happily.

It’s 7am, so I’m going to bed now; first thing I do when I wake up will be start work on giving this app Postgres support; then I can uninstall mysql since nothing else needs it \o/

In further automation, I tired of logging in and opening and resizing xterms, so taking bits from the mythtv wiki, I set up my desktop to automatically log me in and start all my usual programs. Of particular note is xterm’s “geometry” option, which allows you to set the position and size from the command line. Also useful is the “class” option, so that those of us with awesome window managers (enlightenment) can set them to remember position / stacking layer / border style / etc, and assign the remembered values to windows with set classes. For example, whenever I start a window with class “xterm_starter” (ie, by running “xterm -class xterm_starter”), enlightenment will notice it, shrink it, and move it to the corner of my screen, ready to start things from~

Part one of some notes on automation:

Having tired of keeping ~20 shell accounts up to date with my preferred settings, I made a script to do it. Snippets which other people may find useful are:

Only doing things if an app is installed:

is_installed() {
    if type -p $1 > /dev/null ; then
        return 0
    else
        return 1
    fi
}

if is_installed mplayer ; then
    ... do mplayer specific things ...
fi

Outputting large amounts of text from the script into the config file:

cat > ~/.vimrc <<EOD
syntax enable
set autoindent
set autowrite
set nocompatible    " Use Vim defaults instead of 100% vi compatibility
set backspace=indent,eol,start  " more powerful backspacing
set ruler
set foldmethod=marker
if v:version >= 700
    set spelllang=en_gb
    set spellfile=~/.vimspell.en_gb.add
    set spellfile=~/.vimspell.add
endif
EOD

cat > ~/.inputrc <<EOD
"\e[A": history-search-backward
"\e[B": history-search-forward

set completion-ignore-case On
#set editing-mode vi
set mark-symlinked-directories On
EOD

cat > ~/.bash_logout <<EOD
#!/bin/bash

if [ "\$SHLVL" = 1 ]; then
    [ -x /usr/bin/clear_console ] && /usr/bin/clear_console -q
fi
EOD
chmod +x ~/.bash_logout

Now using those, I have all my important settings stored in one shell script. In order to push settings to another host, I can run “cat slek2.sh | ssh user@host.com”, which doesn’t even create any temporary files on the other side of the connection \o/

(Note: the post title, and the script, are called “slek”, which was originally a typo on “skel”, but then got made into “shish’s linux enhancement kit”. Though it’s also tested and working fine on solaris…)

It’s getting people other than myself contributing to it, such that I’ve had to learn how to set up subversion for multiple users :O

A problem I noticed, and google didn’t seem to have the answer for — when the client says “svn: CHECKOUT of ‘[filename]‘: 403 Forbidden ([host])”, and the apache error log says “Access denied: ‘[user]‘ CHECKOUT”, note that authz paths mustn’t end in a slash, unless it’s the root. eg, this works:

[shimmie2:/]
* = r
shish = rw

[shimmie2:/trunk/contrib/link_image]
Artanis = rw

But this doesn’t:

[shimmie2:/]
* = r
shish = rw

[shimmie2:/trunk/contrib/link_image/]
Artanis = rw

I forget exactly when the project was started, but monitoring started a year ago. Thus I shall reminice:

Originally, a bunch of people in an IRC channel I hang out in decided to play Travian. As there were about 20 of us, keeping track was a pain, so I decided to make a map. This first version of the map was an SVG file drafted in Inkscape, edited by hand, and saved as a PNG image.

Pretty soon I got bored of updating by hand, and started looking for a way to automate things. Thankfully, the travian creators decided to be awesome and give out a dump of part of their database \o/ I put it into a local SQLite database so that it could be searched easily. Seeing as this was just a 5 minute hack that nobody other than myself would ever use or even see, I decided to use PHP to write the front end…

After a couple of weeks of adding features, I noticed that it could be useful to other people, so I posted about it on the forums. It ended up being quite popular, and I got several feature requests~ I also added support for other servers, and from there, to other languages. Having people offer to help with translation has been very motivational — It’s nice to know that people care about the project~

Talking about other languages, an interesting note on culture; when an American server is added, I get bug reports along the lines of “Type: Defect // Priority: Critical // The new server was added 3 hours ago, why is it not on travmap yet?!”. When a European server is added, the report is more like “Type: Enhancement // Priority: Minor // There is a new travian server at s2.travian.foo, please can you add it to your list? Thankyou :)”

Another few weeks later I had broken all my web serving records; I was serving tens of megabytes per day, and my server was melting — getting a P2 200MHz box with 64MB RAM to search through 200MB of database several times a second was painful. Switching to highly indexed MySQL made things better, but still bad. I ended up buying a load more RAM, ending up at 256MB. This was OK for a couple of months, until the database grew to be again too large to fit in memory… It got to the point where it was being queried faster than it could respond, so it got slower, and things stacked up exponentially — load average broke 60 within minutes of apache being started :-/

Eventually I gave up on self hosting, and moved across to dreamhost — thanks to the wonders of overselling, they had tons of spare processing power, memory, and bandwidth. Aside from datacenter problems, they’ve been pretty cool.

Since then, growth has been pretty steady. The database is now over 500MB, going over a hit a second at peak times (That’s quite a lot of traffic, nearing a gig a day now…). I also notice that dreamhost moved me to a different database server without telling me :|

Also I figured that with 10,000 visitors per day, there ought to be some way of making some money out of it — putting up a “please pay for my hosting” link earned me a grand total of $5 ($4.50 once paypal took their cut) from one guy — to this day, the only money I’ve made directly from programming ^_^ About a month ago I put some ads on the site to see how effective they were, and since then I’ve made enough to cover nearly half of that 256MB RAM I bought so long ago — another 3-4 months of the same earnings, and I’ll have got as much money out of the project as I’d put in XD

Overall, it’s been a fun year — I’ve learned lots about caching and optimisation, and how much PHP sucks (I *really* wish I’d written it in something else, I didn’t know it’d get so big…); I haven’t actually played travian for a few months now, I’m just sticking with the project because it’s enjoyable~

I forget how I was going to end my rambling, so I shall do so abruptly. End.