The Right Glue

(2008-02-26 19:10:58)

I am a computer scientist by both occupation and interest. When I'm not coding something boring at work, then I'll sure be coding something boring at home. Whenever I get an idea about something cool, I generally do that cool thing until it's done (by which I mean I get bored of it).

One thing I made was Topolife, which I conjured up about a year ago with a desire to implement Conway's Life on various interesting topological surfaces. Topolife gave me hours of math-based entertainment when I first made it. Unfortunately no one else I know shares enough of my particular brand of quirks to see Topolife as the incredible feat of mathemathical majesty that it is. So that project was left to fester deep inside my web server so that only I would ever find it again. But if you want to know how a Conway glider behaves on a real projective plane, then just give Topolife a go! Who wouldn't want to see that?

Back in autumn 2003 when I was a second-year CS student at the University of Waterloo, I was enrolled in CS246 (C++ and software engineering). The final part of the course involved a group assignment designed to apply lots of C++ and software engineering knowledge we as students should have acquired over the course of the term. My group included me, gharbad and qedi. Qedi, being the huge nerd that he is, founded an IRC channel for the three of us. (This channel, #keveX on irc.slashnet.org, is still in use to this day.)

A little backstory: at around this time, a rival Java developer, the Torb, was building an IRC bot designed to take and deliver messages to users even if they were offline. In my opinion, he made the bot a lot less efficiently than it should have been done. I sought to one-up the Torb and produce a faster Java IRC bot.

And so KevBot, the Kevin Immitation Bot, was born.

Initially, KevBot's only reason to be was to be better than the Torb's bot, and also to facilitate sending messages between qedi, gharbad and me. Over time, though, I kept strapping features onto it. After five years, KevBot can now store and play back messages, do floating-point arithmetic, record things people say to play them back later at annoying times, monitor user activity and a whole lot more. KevBot's primary purpose nowadays is to store what I call "factoids". A factoid is any sentence containing either the word "is" or "are". This sentence, for example, is a factoid. KevBot, at the time of this writing, has 66132 factoids stored.

At some point earlier this month I decided that KevBot was not annoying enough, so I decided to use its database of over 66000 factoids as a seed for a Markov chain. For those of us who aren't me and don't remember back to that CS486 (intro to artificial intelligence) lecture on Markov chains, here is a rough explanation:

Suppose you have the book Dune, by Frank Herbert (which is a great book, by the way) and a list of randomly generated numbers. Using the first random number on your list, turn to that page of Dune. Pick a word out of the chosen page. I picked the word "his". Then pick the word after "his", which in this case is "voice". Now, turn to a new random page and look for the word "voice". Make note of the word following "voice" on that page, which in my case is "on". Then look for "on" on a new page and make note of its following word. Continue this until you feel like stopping. You'll end up with a "sentence" like "his voice on to the Baron snapped". The words that appear in the sentence vary with respect to the frequencies of the words used in the source. So sentences generated from the book Dune will resemble the book Dune on a superficial level. Sentences generated from a cookbook will resemble cooking instructions. Any two words go together, but the sentence as a whole doesn't make any sense at all.

Because of this, when KevBot executes its Markov chain algorithm on its database of things people said, the resulting sentences are often uncannily relevent to the conversation at hand, often including the names of regular chatters and their interests. The results can be eerily poetic:

Let's us apart,
under a GO,
crazy and now
23 at the real things
is using your GOD
to be seen;

Sometimes they're kinda deep-sounding:

is the people use the irony, if I live and hunter.

And sometimes they make no sense whatsoever:

it has to baby is so stupid connection reset

Every once in a while, though, KevBot emits a perfectly valid sentence (or close enough) that no one would have expected:

I don't deserve to be retarded

Maybe not, KevBot. Maybe not.

"So long EMAIL program, spread is face in the only available where I gotta do work. And I interviewed with Rip. Until you that are quite normal. Fun in fact so you want. Only works and like a stupid vampires eating your name are right the guys and generally really deadly. Again from the same!"

I enjoy reading output like the above in games like TF2 as a way to help people loosen up. Not many people can continue taking things too seriously in the presence of a Markov chain. And so applied computer science has done its part to make the world a better (and funnier) place!

Comment on this post (3 comments so far)

Latest comments:

Stupid Vampires by KenT (2008-03-04 13:01:35)

it boring that military service by KevBot (2008-03-04 21:59:26)

My wisdom is endless. by KevBot (2008-03-29 01:55:17)

Popularity

By Dean

(2008-02-04 19:07:29)

The Right Glue has undergone an important rite in interactive websites: it has been spammed. To me, this can only mean that my website has a high enough rank on some search engine or spammer hitlist (or however it is that spammers decide whom to spam) to merit being spammed. Someone (or, more likely, somebot) somewhere decided, either consciously or algorithmically, that it would be profitable to fill my site with Cialis ads.

The spam targetted the language trap post specifically, probably because it is the most popular of my individual posts, linked by Gharbad on xkcd's forums. So at least one of my posts is well ranked.

Obviously I can't let spammers have their way with my website, so I have had to put into place some experimental anti-spam mechanisms on the Right Glue. These are some ideas I've had regarding spam protection on anonymous websites that I've never had opportunity to implement. Luckily I finally have a chance!

I want to avoid as much as possible "active" spam protection (for example, image captchas found on many websites) which require users to do something extra to prove that they are no, in fact, bots. I don't really like the idea of treating users as if they are guilty of spamming just because there are some bots out there (hereafter referred to as the Microsoft way), since no one really deserves that. Instead I favour a "passive" protection approach, relying on the fact that spam bots are merely algorithms and have no choice but to follow their instructions.

Specifically, I have done two things to fool spam bots. The first is I've hashed the names of POST variables. Spam bots will look at the names of fields and put certain data into them. For instance, the spam bots put words like "John" into fields containing the word "name". By randomizing the names of the fields using a hash function, bots will not be able to identify what type of information goes into which field. Hopefully this will cause the poorer bots to give up. At the very least, it should help make spam that does make it through easier to identify, since fields will be mismatched.

The second thing I have done is a bit more crafty. I have created a regular field in the comment form called "comment". What spam bot wouldn't want to fill such an obvious target? If the server-side comment script determines that something is in the "comment" field, the comment will not be added. The trick is the "comment" field is completely invisible to humans. Only a bot parsing the HTML would see it and fill it in. That way humans can fill in every field they see without trouble. Bots which do the same will fail.

Given time we'll see if these measures work in practice. I certainly hope they do, but if not I will have to do things the Microsoft way. I'll try not to make them too bothersome. Something like "please input the last two words in your comment into this field" to verify you're not a bot.

Here's to the popularity of the Right Glue! May it be the target of more spam in the years to come.

Comment on this post (1 comment so far)

Latest comment:

Spam Blocking Techniques by KenT (2008-02-08 08:41:23)

Gaming's decline

By Dean

(2008-02-01 19:05:49)

Two new games in the Command & Conquer series have been announced by EA (hereafter referred to as "the retarded monkeys"): Tiberium, a spiritual successor to C&C Renegade and sequel to C&C 3; and Kane's Wrath, an expansion pack to the retarded monkeys' previous game, C&C 3.

Kane's Wrath is slated to have a story that spans the twenty or so years before, during and after the events of Tiberian Sun (C&C 2) and Tiberium Wars (C&C 3), hopefully filling in C&C 3's many wide plot holes. It had been declared that Kane's Wrath will feature CABAL (as seen to the right of this post), Kane's AI advisor gone rampant. CABAL is a favourite among Command & Conquer enthusiasts, and many of us were saddened to learn that CABAL would only have a brief mention in C&C 3's history. The fact that CABAL very nearly succeeded in converting every last human on Earth into a cybernetic slave in C&C Firestorm didn't weigh very heavily on the retarded monkeys' shoulders, I suppose. Kane's Wrath is due to come out in spring 2008, according to Wikipedia, which means we should have delicious cyborg goodies before you can say Computer Assisted Biologically Augmented Lifeform.

Tiberium is a first-person shooter, much like C&C Renegade, which supposedly takes place eleven years after the events of C&C 3. There is very little released so far of this game which is going to be released in the third quarter of 2008. Gametrailers has a gameplay trailer for Tiberium, which seems to indicate that the story of Tiberium revolves around fighting the alien race known as the Scrin. The retarded monkeys have decided to not actually show any of the substance Tiberium in the gameplay trailer, which I find somewhat odd. What is interesting is the nature of the main character's weapon. Instead of have multiple weapons as traditional FPSs are wont, Tiberium instead features a transforming weapon that can morph into different uses depending on need. I think that's a pretty neat idea. We'll have to see how it plays out in practice. The game trailer, which doesn't feature any gameplay, does feature the substance Tiberium. But like most game trailers it's completely useless at determining if a game will be any good. Time will tell.

Since I'm on the topic of gaming I'd like to bring up a disturbing trend I've been seeing in modern games. Recently games have been increasingly hiding content, weapons, maps, features and other gameplay extras from users until they fulfill certain in-game requirements. It's what game creators like the retarded monkeys usually call "unlockable content". What it means is some things are not available to all users of a game from the get-go. They may be superficial things like cool hats to put on your multiple character, or minor things like new weapons or major things like having a different ending depending on the player's skill level. This is fine for things like RPGs, a genre more or less defined by unlockable items, but it's not at all a good thing in FPSs and RTSs.

C&C 3 had unlockable backstory: you can only know the events that lead up the current events if you played certain levels and accomplished certain extraordinary tasks beyond simply playing the game. Many of these tasks are mercilessly difficult to accomplish, and so many players will never unlock the full backstory. Halo 3 has a similar mechanism, except the player can only see the backstory if he's playing on the game's hardest difficulty settings (which, by the way, is incredibly hard). Valve is planning on adding a mechanism to Team Fortress 2 wherein players have to accomplish certain in-game tasks to unlock extra weapons that can be used in normal gameplay. This is only the tip of the iceberg of "unlockable content".

To me, this concept serves only to alienate players. Instead of making a game fun, game makers add in loads of "meta-gaming" so that players feel compelled to play the game more out of a desire to "get that last item". Instead of creating a game that is played based on its own merits (like fun, excitement and entertainment), games are made that resemble purgatorial obsessive chores more than the interactive narratives they're supposed to be. And then there are the gamers who simply can't put the time and effort into getting these extras, and so they feel inadequate and left out. They have to rely on their friends' recounts of the game's "true" (read: hard-mode) end because they can't get it themselves.

In-game metagaming also takes away from those of us who want to play the game for the game itself and not the endless item hunt. I don't really want to play a game which is filled with players who are only there to get some achievement to unlock the next thing. That kind of player doesn't make for good conversation, and he definitely doesn't work co-operatively with others (assuming the game is co-operative in the first place). Why would a game company put its customers in situations like that?

People who enjoy hunting for items and unlocking story through persistence can go play World of Warcraft. The rest of us are rather tired of RPG-like obsessive hunts creeping into real games.

Comment on this post (10 comments so far)

Latest comments:

As if I'd want to by Dean (2008-03-03 18:09:42)

That may be so by Guido (2008-03-28 20:17:54)

So? by Dean (2008-03-28 21:54:54)

This is some kind of footnote. This webpage is awesome and can be viewed in any browser. Even ones that suck ass like Safari and Firefox. Isn't that awesome? This site is best viewed with browsers that aren't maximized on large-resolution displays (> 1024 pixels in width). But then again, if you are running a large resolution and browsing maximized, then you're a terrible person so you don't really deserve to see this site at its finest. Jerk. I mean, seriously. I spend all this time making a nice site and your silly browsing habits ruin its look. That's really cold, man. If you're using IE6, then in order to see the cool avatar effects you need to enable JavaScript. No rights reserved by Dean Whelton (who is awesome) of any of the content, images, design, backend or electrons used in this site. Steal at your convenience. None of it is worth stealing anyway, so there. I have even made an RSS feed for more efficient theft of my intellectual property: CLICK IT NOW!!! Now, don't say I'm not generous. I guess if you want to know more about me, you can visit the about page. It's not really an about page, though. It's just one of the first posts. I don't feel like making a real about page. You can contact me, too. If you feel like it. Are you really wasting time reading this? Go outside or something.