The Right Glue

(2008-04-23 19:20:04)

Metaphors are used by software engineers to help users relate to the software that they're creating. For instance, the name "desktop" for the screen Windows and other systems present to the user comes from the desktop metaphor, which was designed originally to help users understand the nature of the two-dimensional space displayed on the screen. Metaphors are also used by software engineers to talk to one another. Many computer-science terms stem from metaphors, such as structures, trees, tables, indexes, et cetera.

However there is a more overarching metaphor that applies to all software development, encompassing everything from the initial conception of the users' needs to the final, tested and installed product. This metaphor is mostly unused by software engineers when talking between themselves, because for the most part (since it is our profession) we don't need to resort to metaphor to describe what we're doing. The overarching metaphors are used to communicate what might be difficult to do and what might be trivial.

The problem is when you compare software engineering too strongly with its metaphorical counterpart, people start focussing more and more on the tangible metaphor rather than the hokus-pokus of software.

I am going to take this time to describe the current most-popular metaphor used today and explain what problems it has caused. Then I'm going to present a better metaphor and tear it down as well. Ultimately my conclusion will be never to adhere to a metaphor too strongly lest you foster misunderstanding.

Today we use the "architecture" or "construction" metaphor of software development. Indeed, the very terms "software engineer" and "software architect" come from this metaphor. We often say that we are "building" software. Software runs on "infrastructure". This metaphor has shaped the terminology we use to describe these ideas.

The metaphor owes its strength to the age and wisdom of building structures. Humans have been building shelters (or at least finding and improving them) since before recorded history. Though not everyone is a civil engineer or a design architect, we're generally very good at looking a building and deciding whether or not it was well constructed. Even though not everyone can build a house, nearly everyone lives in some kind of man-made construction. Therefore drawing an analogy to construction literally hits home with everyone.

Everyone in the software engineering business knows that spending an extra hour designing a program will save an extra week of coding time. This came from the construction metaphor. Architects and civil engineers spend as much time as necessary poring over every minute detail of the construction plan before even considering moving forward with it. After all, if they get anything wrong the building may not last for hundreds of years. Or worse yet it may collapse and kill everyone inside. Indeed, if everyone approached software design with the idea in mind that flaws could prove fatal then we'd surely try to go the extra mile to make sure everything is perfect.

Because of this attention to detail, the metaphor draws a lot of strength from the higher-ups, whoever they may be. They like to think that we programmers are doing everything we can to construct perfect software that will last forever. And who can blame them?

The metaphor is flawed in many subtle ways that lead to a great deal of frustration when dealing with people very far removed from software development. While I could go on for several pages enumerating them I will stick to the most obvious and problematic.

The concept of "maintenance" as it is used in software engineering is the complete opposite of the equivalent term in civil engineering. Once a building is erected it will slowly degrade by no fault of the engineer who designed it. It's simply a matter of fact. Things in the real world degrade over time. This is not the case with software. If there is a problem being observed with software, it was not "damaged" or "corroded" by some outside force. Rather the one who designed the software forgot that particular detail and no one noticed until now. Software "maintenance" is the programmer changing the logic and design of the program. That is nothing at all alike real-world maintenance.

It takes a great deal of time and effort to change the design of a physical structure once it's built, or even if it's half built. Therefore you have to completely understand what your customers want and tell them, "OK. No more," before the first hole is dug. Nothing could be further from the truth with respect to software writing. New-but-similar requirements and even most new-but-completely-unforeseen requirements can be added to a piece of software in a matter of hours. In fact we, as diligent software engineers, specifically design our software in such a way that adding, removing and changing details is as painless as possible. Now, obviously there is a limit to how many and what kind of changes that can be made to a design without needing a lot of effort, but that threshold is much, much higher than most would imagine.

This misunderstanding of software is so pervasive in the minds of non-software engineers that often teams are specifically told to go to great lengths to avoid "hard coding" things into a system. The act of avoiding hard coding anything is called "soft coding" and it is nothing but trouble. Nothing is more prone to error and bad design (and, ironically, inflexibility) than soft coding, at least as far as I'm concerned. The separation of logic (i.e. code) and what the logic works with (i.e. data) is good enough to guarantee the least amount of effort needed from each party involved (most of the time).

To reiterate my point (which I really can't stress enough), consider the effort needed when I added a completely new piece of functionality to KevBot. KevBot was never designed to do anything remotely similar to a Markov chain. Its data structures aren't designed for the deep searching required to build it. I had never written a Markov chain algorithm of any kind before. In fact, we had only covered them briefly in my intro to AI course in university. It took about an hour, maybe an hour and a half, to get it up and running and live and interacting with users. Same deal with this website. About two hours from having an idea to having a full site written up in a language (PHP) I had never used or seen before.

Now, I'm not boasting my ability or claiming that KevBot or this website are at the same level of complexity as most business applications; rather I'm claiming that software is highly malleable, with previously inconceivable features added in mere moments. I'm claiming that had one adhered to the architecture metaphor, one would think that this kind of thing would take weeks or even months to complete. That is because the metaphor is flawed.

A better metaphor for software engineering that clears up these important misconceptions is Lego. Lego, the children's toy of colourful little blocks that fit together is more similar to software development than architecture. And I can prove it.

First of all, given enough of it, Lego can build anything. Right there, Lego is more robust than civil engineering. Name everything you can do with a house. You can live in a house, and that's pretty much it. Everything else you do in a house is done with supplements to the house itself like beds and televisions. Civil engineers don't build beds and cool robots. But you can build beds and cool robots and much more out of Lego. Just like how you can do more than just build websites with software. Software and Lego can both do anything.

Lego requires careful planning, in fact all of the careful planning that civil engineering requires, to make something right. Just like construction, every hour spent planning saves days once you start snapping the pieces together. Lego blocks also don't degrade over time. If you build something out of Lego and it stops working, then it's because you didn't squeeze the pieces together tightly enough, or you didn't build it structurally sound enough to keep from falling apart during use. If it breaks, it's because it was always broken, just like software.

Anyone can use Lego. You don't need a degree in Lego science to build a rocket ship with Lego. You don't need 15 years of experience designing cool Lego toys in order to make a neat robot. Just like programming (and very, very much unlike civil engineering) Lego is very easy to pick up. (Not that this is necessarily a good thing, since you generally don't want to set up a production Lego construction that was assembled by someone who's never used Lego before. Just like software design!)

Changing a Lego construction is easy, too. You want to add a third arm to your cool Lego robot? Just snap it into place! You don't like this colour of block? Take it out and put a new colour in. It's that easy. Just like software.

But even this metaphor has issues. Building an algorithm isn't anything like snapping colourful bricks together. It's not that easy. Also, the importance and difficulty of debugging is completely lost in this metaphor. It's generally very easy to see what part of a Lego robot is flawed by looking at the broken wreckage. Software not having physical form makes this much, much more difficult. Lego also requires a surplus of bricks to create something, often requiring the cannibalization of previously-built cool Lego robots to get them. Software has nothing like this. Code has no physical form, after all.

No matter how cool Lego is (it's very cool), it can't serve as an adequate metaphor for software development. It's not Lego's fault, though. This is the problem inherent in all software metaphors. Just like how no one has a physical "taskbar" or works by moving "windows" around on his desk (the "desktop" metaphor). No one metaphor is going to fully explain any discipline. It's not that we shouldn't try to use metaphors to help describe what we as software engineers do, it's that we shouldn't let these metaphors drive the nature of the discipline.

Software engineering is software engineering. It's not civil engineering, it's not architecture, it's not Lego. Never forget that.

Comment on this post (2 comments so far)

Latest comments:

a slight problem by Jessica (2008-04-23 21:09:11)

Hah by Dean (2008-04-23 22:00:41)

Gehn

By Dean

(2008-04-17 19:40:29)

Let me tell you the story of a man named Gehn (and link to Wikipedia excessively in the process).

In the Myst series of games, there is a civilization called the D'ni. In their vast underground caverns, the D'ni created a way to "link" to different worlds and civilizations using a precise system of writing.

D'ni civilization was based around various guilds. One of these guilds produced special books, one guild made special ink, one wrote worlds into the books using the ink, and one verified that the writers were doing a good job.

Using the precise language of the writer guild, the correct ink from the ink maker guild and the right book from the book maker guild, a D'ni could create an entirely new world, complete with flora and fauna, and visit it. Each book was a work of art, bringing its passenger to the imagination of its writer. (Technically the books didn't create new worlds, but rather "connect" to existing worlds that match the precise descriptions in the books — though most of the time there is no distinction between creating and connecting.)

If the writers are careless, then a simple out-of-place word could mean the destruction of the world on the other side, and everyone living there. That is why only the writer guild was allowed to create worlds, and even then only under the scrutiny of the guild masters. If even one feature of the world was described too vaguely, then anything could happen. For instance, the planet's star could go nova, or the planet's weather could be too hostile to support life. When playing god anything and everything can go wrong.

Without going into the details of the games' stories, the D'ni civilization fell leaving a young would-be ink maker named Gehn to fend for himself in the ruins of the once-proud D'ni capital. With access to countless left-over books and inks, Gehn took it upon himself to bring D'ni back from the grave by writing new worlds and teaching the people there the ways of the D'ni.

But Gehn was too proud of his heritage. He didn't take the time to fully comprehend what he was doing, believing his D'ni blood would show him the way.

He started writing books.

Since he didn't have a firm understanding of the creativity involved in creating worlds (nor the immense level of detail and precision required to keep them stable), he never wrote a sentence of his own. He simply "copied and pasted" sections from books still scattered around the remains of the capital, having only a rough idea of what each one was meant to do.

Needless to say, the worlds Gehn wrote all ended in ruin. Each one ripped itself apart in one way or another. Even his best world, Riven, was doomed to be, well, riven. Because the passages he stole from other books were a small part of a greater whole that Gehn never cared to understand, they did not work out of this context as well as they should have. Every spliced citation caused a new problem in Gehn's worlds.

Gehn embodies a rather disturbing fact of life, which is that people who don't know what they're doing are incapable of knowing that they don't know what they're doing, and indeed believe they are doing the right thing. Gehn truly believed that he could bring back the D'ni civilization, and that he was doing the best job possible, even though everything he created was riddled with flaws.

That sounds a lot like software engineering, doesn't it? We have programmers who create, infrastructure guys who give the programmers the tools they need to create, and testers to make sure the programmers' creations don't kill everyone. And just like Gehn we have countless people who believe that programming is nothing more than taking code written by other people without taking the time to fully understand it (or, as it is in this case, without even bothering to read it in the first place). At least as long as this is the case we can still laugh about them and their silly beliefs on such sites as The Daily WTF.

The lesson today is don't be Gehn, lest all of your creations crumble in front of your disbelieving eyes. Don't put yourself in a situation away from people who can tell you that you're doing something incredibly stupid and naive. A little bit of self doubt can go a long way; and it's much better in the long run to have asked someone for advice and succeeded than to have pushed forward stubbornly toward your own doom. The worst thing you can possibly do is assume you know everything.

Comment on this post (3 comments so far)

Latest comments:

Comment. by Abe (2008-04-19 01:04:08)

Once more to wiki once more. Hope it comprehendable. by Baronen (2008-04-20 20:40:17)

Dean. Save my purgid Text. by Baronen (2008-04-20 20:51:37)

Kane's Wrath

By Dean

(2008-04-05 11:36:55)

So on April 1, at around 0830 hours ADT, a guy named Doug hacked into my website and converted it into a LiveJournal. Luckily I was able to save the day using regular expressions!

Now that the world is safe (and blue) once again, it's time to write a post about C&C 3: Kane's Wrath.

I built my computer, Diaeresis (photos available), specifically for the game Command & Conquer 3: Tiberium Wars. Well, that's not entirely true. I was planning on purchasing a new PC in 2007 in order to play the games Bioshock and Portal, which were due out in the middle of the year. Then I heard news of C&C 3, yet another game in Westwood's original (and amazing) series of games. Even though the retarded monkeys had long ago purchased Westwood, dissolved most of its employees and crushed all creativity within a 10-kilometre radius of its headquarters, I somehow remained hopeful. So I built Diaeresis about a half a year ahead of schedule.

Needless to say I was disappointed by C&C 3's terrible plot hole-heavy story and decidedly un-Command & Conquer-like gameplay. Somehow EA with its many simian minds came to the conclusion that Tiberium (a toxic yet valuable substance that science-fictionally is eating away at the surface of the Earth, transforming it into something inhospitably alien) should be made into an inorganic crystal, instead of its previously almost-alive look. That's definitely my biggest gripe with the game, and indeed the source of the majority of the game's plot holes.

At the end of C&C 2: Tiberian Sun, Tiberium was in the process of converting all unprotected life into horrific monsters. During the events of C&C 2: Firestorm, very little recognizable life remained on Earth. Nearly all of it, be it plant, animal, terrestrial or aquatic, was converted into a Tiberium-based equivalent. Tiberium itself spread like a plant, indeed mutating terran plants to suit its proliferation.

But then suddenly in C&C 3, Tiberium is just this mostly inanimate crystal. The Earth itself is mostly fine. There are no horrible mutants flying around, no Tiberium-based life at all. Suddenly there are vast regions of the Earth that are not at all exposed to Tiberium. Suddenly the oceans aren't teeming with Tiberium to the point that ships can't move at all. It's as if C&C 3 weren't based in the same world as C&C or C&C 2 at all.

This and many other plot holes were not resolved in Kane's Wrath. (Ha! I bet you were expecting me to say the opposite of that!) I'm just as disappointed with Kane's Wrath as I was with C&C 3. The story itself was interesting, since everything happens from the point of view of Kane, the primary antagonist in all C&C games. Kane is a very charismatic and intelligent character, so it's always a good thing to see him on screen. Kane's Wrath did not fail to deliver.

Apparently realizing that C&C 3 was full of holes, EA made the story of Kane's Wrath take place before, during and after the events of C&C 3. This allowed them to fill a few of the less glaring holes, such as what happened to certain characters behind the scenes, or indeed how Kane managed to survive his own death in C&C 2. Some of the events in Kane's Wrath fit so well into the main story of C&C 3 that it almost makes me wonder whether EA left plot holes there on purpose in order to magnificently fill them in later. Honestly, though, I don't give them that much credit.

Kane's Wrath was fun, albeit short. I found out that the computer cheats in the later levels, and that really ruined the experience. But its story was good enough and fit disturbingly well into the main story, so I was definitely entertained by it. That is, afterall, the point of games.

Gameplay is exactly the same as C&C 3, except with these new special units called "epic units". All of the epic units are terrible wastes of money except for the GDI epic unit, which is utterly overpowered compared to the other two. I don't know how that slipped through the cracks, but maybe there will be a patch to balance things out.

If you liked C&C 3, then you'll like Kane's Wrath. It's exactly the same.

Comment on this post (1 comment so far)

Latest comment:

Global conquest mode by Dean (2008-04-07 07:44:08)

Feelin' emo

By Doug

(2008-04-01 08:25:36)

Currently listening to: Touhou GST

Dear LiveJournal,

I was feeling sad, and I was feeling lonely. So I decided to go down to Wal-Mart on the weekend to end these dark times of my life by filling my computer with software provided by the retarded monkeys. Those monkeys were hard at it again making a new game, just for me. I'm not going to tell you what it's called or link to it or anything. Just that it's about a man named ~~Kane~~ Jeff and his ~~wrath~~ dirigibles. Yeah. Jeff's Dirigibles.

So I drove Jeff's Dirigibles around for a while, dropping bombs on everyone and everything but then suddenly Jeff gets him away from something I'll make fun of the marking ship is basically what started out in the whole comic if it looks like obviously drawn further jeaportizing the first 150 ARE showing her penis is not VERY very attractive from the desktop 3-D scene is it becomes implies What wake to go! I'm really happy about the 3-D scene because all of Jeff's Dirigibles look really well modelled.

Still listening to: Touhou GST

Later in the day, I played some drunken Guitar Rock Hero Revolution at the party. You know. The party. It was awesome. The girls were doing the non-video. I could have a requirement for their four feet all night long, if you know what I mean. Unfortunately that dick Todd came out of nowhere and stole the show with his Xbox. I guess in the end I can't really compete with Xbox. Not even after I showed them my Longcat.

That got me pretty depressed, which is why I bought the aforementioned game Jeff's Dirigibles. Dirigibles are cool. Me and the guys take Steve's dad's dirigible out on the town all the time. I love it when we pick up fly honeys (bees, that is) in that thing. I love all of my friends.

Except Jim. He's not even my Facebook friend anymore after what happened at Todd's Guitar Rock Hero Revolution party. Can you believe he actually fit all of that into his mouth? Pshaw. I know, right? C'mon. You gotta be kidding me.

Speaking of rodents, Janice needs a new source of mice to feed her pet snake since the pet store got the restraining order (thank you, Jim!). If anyone knows where she can get some, leave a comment up ons so we can hook Mr. Slithers up with some chow.

So as I was saying, at Todd's Guitar Rock Hero Revolution party was awesome. I got like a triple high score. I even beat Nick's high score. That was the highlight of the evening. Nick was right there, too, bragging about his bitchin' score to all the ladies. I whipped his ass what how! Zam! I was feelin' pretty good 'till Todd brought out his Xbox. The bastard.

So I was like, "Fuck that shit." And I took Nick's beer and drank it down and ran the hell out of there. High-tailed to the next city with my dad's dirigible. I think Matt was driving. I don't remember much since I was too wasted. I think someone must have slipped something strong into Nick's beer.

I don't remember much after that, but I woke up with the taste of stick tack in my mouth with my shirt on backward in ditch. Dunno where Matt went.

In short, it was the best $30 I ever spent.

Doug OUT

Comment on this post (4 comments so far)

Latest comments:

hey by jim (2008-04-01 09:12:48)

that's it by jed (2008-04-01 10:44:48)

What? by Doug (2008-04-01 12:13:25)

This is some kind of footnote. This webpage is awesome and can be viewed in any browser. Even ones that suck ass like Safari and Firefox. Isn't that awesome? This site is best viewed with browsers that aren't maximized on large-resolution displays (> 1024 pixels in width). But then again, if you are running a large resolution and browsing maximized, then you're a terrible person so you don't really deserve to see this site at its finest. Jerk. I mean, seriously. I spend all this time making a nice site and your silly browsing habits ruin its look. That's really cold, man. If you're using IE6, then in order to see the cool avatar effects you need to enable JavaScript. No rights reserved by Dean Whelton (who is awesome) of any of the content, images, design, backend or electrons used in this site. Steal at your convenience. None of it is worth stealing anyway, so there. I have even made an RSS feed for more efficient theft of my intellectual property: CLICK IT NOW!!! Now, don't say I'm not generous. I guess if you want to know more about me, you can visit the about page. It's not really an about page, though. It's just one of the first posts. I don't feel like making a real about page. You can contact me, too. If you feel like it. Are you really wasting time reading this? Go outside or something.