How the hell do I write stuff? It just comes out, like poop, and the result is usually nearly indistinguishable.
Brad Pitt, in an interview, once said something like: "I don't know what the hell I'm doing up there in front of the camera. Really, I have no f---ing clue." He's become quite an actor, hasn't he? Maybe you've decided he'll never be Bogart (he's got a few years to go, though, doesn't he?), but you have to admit he's a lot better than various actors-who-always-play-themselves whose names rhyme with Rom Ruise.
Sorry, didn't mean to imply even indirectly that I'm a good writer; I know better. Don't ask me how I know — I'd have to think of seven words in the right order to explain it. I just know. I suck.
I do read my old entries occasionally, though, and I think: "Gosh, how the hell do I write that stuff? I could never do that! Well, er, I mean now, that is. Obviously I did it before. Just what are you trying to say, anyway? Do you think I'm some sorta dumbass?"
I wonder.
Dave
I learned to write from my brother Dave, who as far as I can tell wasn't a very good writer. He was a good Liver, to coin an utterly awful phrase; he lived, and that's always better than writing. Writing just tries to show you how people lived, and it never comes close, never gives you that rush of being there just when something wonderful is happening.
Dave died, you know. I've often thought that if I ever wrote a book, it'd be about Dave. He just kinda got sick one day. We golfed, we biked, we had long late-night discussions about his philosophy class, we reminisced about things you could never repeat, during the years we lived in different states, stuff happening mostly at parties, teenager sins you committed as a teenager. Fun stuff. Nothing you didn't do yourself, of course, but it's still unprintable.
He died. I've been dead ever since then, you know. I didn't close my eyes for a single night, for four years after he died, without thinking of offing myself, becaused I missed him so much. My brother Mike knows; my brother Kevin knows; my Dad and Mom both know. We were all there. It's like losing a limb, but worse. So much worse. I realized after Dave died that I'd glady have given both arms and both legs to have him back. I would.
Everyone loved Dave. Everyone wanted to be Dave. Dave had wine-tasting parties at his little apartment; he taught us all to golf and to keep aquariums and to mountain bike and to appreciate the fine points of football strategy. He had a soul-beagle, Bentley, who still misses him to this day.
He had a girlfriend, Nancy, who was more of a wife to him than any wife I know. My wife agrees, and she's only met Nancy once.
Dave just got sick one day. It happens. He didn't feel good. He was coughing. He had night sweats. You know, a cold. A fever. Bronchitis. Something nasty. Didn't want to get anyone else sick — I remember he would sit away from us on the couch so he wouldn't give it to anyone else. He was so considerate.
We didn't know it was a tumor. He was only 23; I mean, come on. He was an all-star football player in high school. You don't get tumors when you're 23.
His doctor mis-diagnosed him, twice. It's OK; we forgive him. It's been 8 years, and I think we all forgive him now. I had a friend in the Navy, back when I was in the Navy, who got mis-diagnosed. It happens. His doctor said he had, I dunno, a cold, something lame. He had leukemia. Doctor mis-diagnosed him twice, just like Dave. With Dave, the doc said it was bronchitis, nasty case, definitely needed bedrest.
The Navy friend got heli-lifted off the sub to a medical facility in Oregon. He didn't make it.
Dave didn't get to ride in a helicopter, but he did get an ambulance. On the 3rd visit to his doctor, the doc took his oxygen level and said: "Ooh, time for the ambulance." Dave trusted him. Turns out Dave had lymphoma, and pretty advanced. No reason. No family history. He just had it. Maybe the environment. Maybe too much coffee or nutra-sweet. Nobody knows. And like everything else in Dave's life, his lymphoma was world-class. It ate him up in a way that no other cancer could.
You really don't want to hear about that part. Trust me.
We trusted Dave. He had a sense of humor like you've never, ever seen or heard before. It would take up several chapters in my book. Dave could make people laugh who had obviously not laughed in years; he'd crack a joke, something made up on the spot, context-sensitive and all, and they'd HAW, HAW, *hack* *cough* HAW HAW HAW until we thought they were going to have a stroke. Dave was the only person I knew who could make someone laugh so hard that I thought they were physically uncomfortable.
So it goes.
I'm not going to make my blog about anything at all. You can peg me as a programming-language guy, or a would-be math guy, or an inconsiderate jerk who says bad things about the vehicle with which you earn your living. But that's not what I'm about. Because I appreciate that you're reading this, I really do, but I'm not writing it for you. I don't know you.
I'm writing for Dave. I sure miss him. We all do. Everyone from Geoworks misses him. They made him a big banner, back when we knew he was sick, but we didn't know it was that bad. I mean, I should have known. The first day in the hospital, after his ambulance trip, after he passed out from lack of oxygen at community college from climbing six flights of stairs from a broken elevator, with a tumor the size of your fist growing between his heart and lungs, and the nurses asked him what he'd been doing that day, and he said he passed out trying to go to class, and he said their eyes got all big and round, I should have known.
Because his doctor started crying. I've never seen that before, and I hope to God I never see it again. She was the visiting doctor, the resident, whatever they have at 10pm at Swedish Hospital in Seattle, Washington, the place Dave spent the next year and a half, the rest of his life. She looked at his chart that first day, and said some encouraging words to him, and then as we were walking down the hall, me and my mom and this strange doctor, she was crying. We thought that was kind of weird, because you don't cry in public, especially if you're a doctor, especially hanging around the family of someone you just saw.
Dave made so many jokes that we couldn't even understand them all, there in the hospital. The nurses loved him. One nurse told him: "You were the best! Even Dr. Wasserman says so!" Dave kidded her for the next four months over that one. "Don't refer to me in the past tense!" I saw her blush every time he mumbled it through the morphine. But he knew and she knew and I knew that he was just messing with her, just having fun on his deathbed. Who else can do that? Not me, I don't think. I don't know.
I remember a road trip, one we did in our parents' van way back when, and our little brother Kevin was about seven years old. Dave taught Kevin to say: "Psh! Whatever!" It took him a little while to grok the concept. The idea was that any time someone said anything you disagreed with, or even if you just felt like it, you would reply: "Psh! Whatever!" You could substitute the socially acceptable variant "Tsk! Whatever!" as long as you could produce a suitably sardonic clicking sound with your tongue, a sound to make Zulu heads turn in envious surprise. Dave had mastered the depreciating tongue-click. For mere mortals, the acceptable default was "Psh!"
My stepmom Mindy was less than pleased. "Kevin, don't you listen to them!"
You wouldn't believe the cheering that me, Mike and Dave produced at 7-year-old Kevin's beautifully crafted response: "Psh, whatever!" (Hi, Min!)
Dave imparted me and everyone near him with a sense of humor, by osmosis, although we were all really just a pale shadow. He had his world-class sense of humor until the very very end. A few months before then, I was visiting him in his hospital room, and he told me in thick, steroid-induced tones (after having thrown up his esophagus the night before, which he recounted to me with some surprise as being like spitting up long filets of salmon) that he'd lost bowel control a few days back, because of the chemotherapy, and he'd had an incident "like the one in Trainspotting". I hadn't seen Trainspotting at the time, but I got the picture. I didn't know what to say, so he chimed in, almost unintelligibly:
"Look on the bright side: at least I didn't have to clean it up!"
How could I laugh? How could I not laugh?
I'll say some pretty strange or seemingly mean things in my time, in my blogs, but you have to keep it all in perspective. My brother was tortured to death. I'll spare you the gruesome details, but aside from the miracle of morphine, those folks in medieval torture chambers had nothing on him. His suffering lasted 18 months, during which he basically dissolved, for all intents and purposes, and in the end I think (not really knowing, myself, but guessing) that the worst part was psychological. Facing your own death at 23 years old is pretty scary. Especially when you're melting.
So I probably have a slightly different perspective than you do. To me, it doesn't matter all that much anymore. I just try to make people around me happy, and enjoy myself, until, you know, I have some sort of major Trainspotting incident. Hopefully one that I don't personally have to clean up.
I don't really mean to be mean, though. I hope you realize that.
Stuff
Despite my best intentions, my blood pressure occasionally rises when I blog. Or more precisely, after I blog, because no bowel movement is ever inspected as scrupulously as the articles posted to Reddit. Even when my blog tries to be innocuous, the comments always seem to get to me.
My doctor says I might have bronchitis. What the hell do they teach med school students, anyway? It's as bad as a Computer Science degree.
I wish there were a way to request, respectfully, that certain of my articles not be posted to Reddit, because even though I want people to read them, I want them to read them at the right time. And that's different for everyone. Probabilistically speaking, the right time for most people is not going to be the day after I post them.
Then of course there's Digg, the Reddit for... Digg people, I guess. Diggers. Duggers. Whatever! Lord help you if you get Dugg, or whatever it is they do over there. And del.icio.us, which in addition to being hard to type is no longer the sprightly young company it once was, ever since You-know-hoo! bought them. Sometimes I wish I could just never be posted there.
That's not nice to the folks looking for karma, though, I guess, so really I just mean this entry, today.
I'd like to blog more about non-technical stuff. I feel like blogging about technical stuff is, well, you know. Dirty? Incestuous? It's not like I'd be saying anything you don't already know, or won't already know at some point, from someone else.
On the other hand, I always feel the (few) bloggers I read ought to stick to the same topics. If I'm reading Bill de hÓra, the only blogger I read regularly, and he suddenly starts talking about his dog, then I feel ripped off, as if the Discovery Channel had started doing a chef competition, or the Food Channel started doing specials on wrestling alligators.
That's not entirely fair of me, I know. People are always broader than what they blog about, but we sort of expect the best bloggers to stay on topic, to keep blogging about whatever we liked last time we read them.
Well, I'm going to take a deep breath, a leap of faith, and see if I can broaden my blogging to include non-cs-technical topics. Yes, some people will whine and moan about it; people will whine and moan about anything and everything. I do it too. But I'd love to blog about the movies I like, and video games, and music, and books, and people, and just plain old good times I've had. Because you never know how long it'll last.
I'd like to ask you, just one person to another, not to post this blog entry to Reddit, Digg or similar. I'd prefer that people learn about my brother Dave through some mechanism other than a newsfeed full of karma modders. You know? So I've deliberately avoided tech topics in this entry, in the hope that it will somehow pass unnoticed.
We'll see.
If not, well... psh! Whatever.
Miss you, Dave.
Rabu, 19 April 2006
Sabtu, 15 April 2006
Software Needs Philosophers
Software needs philosophers.
This thought has been nagging at me for a year now, and recently it's been growing like a tumor. One that plenty of folks on the 'net would love to see kill me.
People don't put much stock in philosophers these days. The popular impression of philosophy is that it's just rhetoric, just frivolous debating about stuff that can never properly be answered. "Spare me the philosophy; let's stick to the facts!"
The funny thing is, it's philosophers who gave us the ability to think rationally, to stick to the facts. If it weren't for the work of countless philosophers, facts would still be getting people tortured and killed for discovering and sharing them.
Does it ever strike you as just a teeny bit odd that after a brief period where philosophy flourished, from maybe 400 B.C.E. to ~100 C.E., we went through a follow-on period of well over one thousand five hundred years during which the Roman Catholic Church enslaved everyone's minds and killed anyone who dared think differently?
What's weirder is that we tend to pretend it didn't really happen. We like to just skip right over the dominance of religion over our minds for a hundred generations, and think of religion today as a kindly old grandpa who's just looking out for us kids. No harm, no foul. Let bygones be bygones. Sure, there were massacres and crusades and genocides and torture chambers with teeth grinding and eyes bleeding and intestines torn out in the name of God. But we were all just kids then, right? Nobody does that kind of thing today, at least not in civilized countries.
We try not to think about the uncivilized ones.
It was philosophers that got us out of that Dark Ages mess, and no small number of them lost their lives in doing so. And today, the philosophy majors are the butts of the most jokes, because after the philosophers succeeded in opening our minds, we forgot why we needed them.
And if we stop to think about it at all, we think that it was other people, people who are very unlike us, who committed those atrocities in the name of Faith (regardless of whether it's faith in a god, or in a political party, or any other form of mind control carried out by force).
We like to think we live in an enlightened age, but we don't. Humans haven't changed significantly in 10,000 years. We're still killing and torturing each other. It's apparently incredibly easy to decide to kill someone and then do it. Happens every day, all around the world. Torture, too.
But those people are just people. If they had been born down the street from you, they'd have gone to school with you, been friends with you, learned to program with you, written blogs and comments, never tortured or killed anyone in the name of an idea. They'd have been you. Which means they are you; you just got lucky in where you were born.
One of the commenters on my last blog entry expressed the fervent wish that I drop dead. To be sure, they qualified it with "on the internet". But if they really feel that way, especially about something as hilariously and absurdly unimportant in the Grand Scheme as whether the Lisp programming language has any acceptable implementations, then what does it say about us?
Everyone who commented angrily on that blog entry was caught. I caught you, anonymous or not, being a religious fanatic. The only "negative" commenter who doesn't appear to be a religious zombie was Paul Costanza (ironic, since he claims to be the opinionated one), who relegated his comments to pedantic technical corrections. They're welcome, of course; I'm always looking to correct any technical misconceptions I harbor. But they're moot, since even if I was wrong about every single technical point I brought up in that entry, my overall point — Lisp is not an acceptable Lisp — remains largely uncontested by the commenters.
Some of them just don't get it, which is fine; no harm in that. If you've been using Lisp for years and years, and you've written books and articles and zillions of lines of Lisp code, then you're unlikely to remember anything about what it's like coming to Lisp for the first time. They're religious because they've forgotten what it's like to be a skeptic.
But make no mistake; a substantial percentage of people who take a side in any programming language discussion that devolves into a flamewar know exactly what the other side means, and they want to invoke the Ultimate Censorship: drop dead! Killing someone, after all, is one of the best ways to silence them. You also have to burn all their writings, which is getting harder these days; hence the increased vehemence on the 'net.
Those of you who've followed what I've written over the past year or so know where I'm going. I'm taking a stand, all right, and it's a very definite one. I'm finding myself drawn inexorably towards a single goal: stamping out technological religion, because I'm frigging tired of not being able to stick to the facts.
FACT: Java has no first-class functions and no macros. This results in warped code that hacks around the problem, and as the code base grows, it takes on a definite, ugly shape, one that's utterly unique to Java. Lisp people can see this clear as day. So can Python folks, so can Ruby folks. Java people flip out, and say "macros are too much power", or "what do u mean i dont understand u" or "fuck you, you jerk, Lisp will NEVER win".
You think I don't hear ALL that, and much more, in the hate mail I get every day?
I sure wouldn't want to be alone with a Java fanatic in a medieval torture chamber, because God only knows what they're capable of.
Turn the mirror towards Python, and what happens? Funny, but the Java folks will mail me saying: "yeah, I've always known I detested Python, and you really nailed exactly why. Thanks!" Meanwhile, Python folks are literally frothing at the mouth, looking for the "Kill That Bastard" key on their 101-key keyboards.
I turned the mirror towards Lisp yesterday. Had to go to the bathroom like nobody's business, and my wife was expecting me home any minute, so I rushed it out: just a few thoughts here and there. So the Gorgon only caught the tiniest glimpse of itself, but hell evidently hath no fury like that of a Lisper scorned, and all that.
It doesn't matter that I rushed it out. I'm glad I did; spending any more time on it, trying to get it "right" by looking up useless factoids like how you can override length's non-polymorphicness with some weird setting (when it plainly should just be the default), would have had the exact same net effect: Lisp zealots would have found some way to turn it into a flamewar. And I'd have been out 2 or 3 more hours.
Let's call it a troll, then, because it was poorly researched; it was just some months-old recollections of pain I'd gone through last year trying to commit to Common Lisp, after another year of trying the same with various flavors of Scheme and finding them all wanting. As far as I'm concerned, Lisp is unacceptable today; it's my opinion and just that, but I'll stick with it.
I still need Lisp; after you learn enough of it, it becomes part of your soul. I get my fix hacking elisp, and I do a lot of it. The commenters are quite right; I've never written anything substantial in Common Lisp, because in each of my serious attempts, there was too much friction. Risk/reward wasn't high enough, and believe me, I wanted it.
But after many attempts, I've given up on Common Lisp. They won't let me use it where I work, and there are probably more Lispers per capita where I work, including some famous ones, than at any other big company in the world. If we can't use it where I work, then it's frigging unacceptable; that's the shortest proof I can offer.
What I'm far more interested today is the situation that arises if you consider my post a troll. I'm far more interested in the social consequences of working in a world filled with religious fanatics of different religious persuasions. Especially given that it's a world in which "natural religion" has, by and large, been marginalized through the work of philosophers.
Let's look at this world in a little more detail, starting with Peter Siebel's comment, which I believe is the most interesting. Peter said:
His first sentence speaks volumes about the sociology. His viewpoint is exactly what they teach us all as kids: If you don't have anything nice to say, don't say anything at all. We like to think people have a right to believe whatever they want, and that it's not nice to say mean things about other people's beliefs, especially when their livelihoods are at stake.
That's where philosophers come in, folks. They pick your beliefs apart and show you in unforgettable ways the consequences of what you believe in. I'm no philosopher; I know basically nothing about it, but I can tell you I wish fervently that some great philosophers would come along and effect change in our technical society.
Because if nothing else, I can see the consequences of the way we're thinking about things. One of many such consequences is that languages aren't getting any better, and the worst offenders are Lisp and Scheme, which by rights should be racing along the innovation curve faster than their supposedly less capable peers. But they've stagnated worse than any other non-dead language I can think of.[1]
Programming languages are religions. For a long while now I've been mildly uncomfortable calling it "religion", but I don't feel bad about it anymore. They're similar enough. At the top of the language religion is the language itself; it serves as the deity and the object of worship.
Like any other organized religion, there's always a Pope (or a politburo chairman, in countries where the government has brutally set itself up as what is for all intents the religion of choice): a spiritual leader that gives the religion the human touch. This person is almost always the language designer, of course. In Lisp's case it's complicated, because McCarthy, Sussman and Steele aren't very active as spiritual leaders for their languages anymore.
Every major organized religion is a heirarchical government, and programming languages are no exception. You'll find equivalents of cardinals, bishops, priests and laity in programming language camps: the closer you are to the fire, to the spiritual center, the higher your rank. It's a great way to quantify your perceived self-importance: a high-score list, in effect. Great for the ego, but it makes you a piss-poor debater, because you're so emotionally invested in your status.
You'd think your rank would be accrued by virtue of your technical and/or documentation contributions, but in practice it's usually more of a function of how many converts you've gained, how many followers you have, how much you've been spreading the Word.
That's why Paul Graham isn't the Pope of Lisp. He's eminently qualified, but unfortunately he's a heretic. Notice that almost none of the commenters on my last blog mentioned the PG argument I made. The only one who did (as of this writing) tried to make it an argument for Common Lisp. Let's face it: you can't give those heretics too much press; people might start listening to them!
Peter, are you beginning to understand why I write so much about something I apparently don't like? It's because I wanted to like it but found it fatally flawed, technically and culturally. It's as if I were a would-be convert to Roman Catholicism, but I can't bring myself to commit because I've seen too much of their role in creating a history that ironically we all wish we could rewrite.
I was born and raised a Roman Catholic, and I renounced it when I was thirteen years old, after my Uncle Frank (a devout terrorist Catholic if there ever was one) told me to stop reading the Bible, that it would "really screw a person up" to do that, that you needed someone to interpret it for you. That wasn't the only reason I renounced it, but it'll suffice for our purposes.
Technologically I was born and raised an assembly-language programmer; at least that's what my first real job was, for 5 years after I got my CS degree. Assembly is just flagellation, though, and damned uncomfortable at that, so I joined the Church of Java for fully seven years. And practically at the very moment I'd finally tired of chafing at Java's limitations, Paul Graham came along and through his early essays, showed me Lisp. What a great new religion!
Problem is, each time you switch religions, the next one has less impact on you. Once a Catholic, always a Catholic, they say. I don't know what that means for me, since I was raised by the assembly-language wolf, but it appears to mean that I'm never going to be enthralled with another programming language. And now that I've swallowed the red pill, what choice do I have? I need to try to show people what's out there.
Interestingly, it was Peter Siebel's most excellent book, Practical Common Lisp, that played the role of Uncle Frank and killed my desired to continue with Common Lisp. Peter was the first person to show me beast's underbelly. Every other Lisp book had pretended it was pure and beautiful and uncorrupted, because they left all the nastiness out as "implementation-defined". Once I saw what you really need to do in order to build something resembling a portable Lisp code base, and then had a few runs at it myself, I threw in the towel.
I much prefer Lisp the idea to Lisp the implementation.[2]
I can tell you this: I've tried writing this essay for a year. I've tried fully a dozen times. I've tackled it from a dozen angles. I've wanted to say it — software needs philosophers! — so many times, in so many ways. We need great thinkers — the Fyodor Dostoyevskys and David Humes and Aristotles and Jean-Paul Sartres and Ben Franklins and Galileo Galileis and Bertrand Russells and Albert Einsteins to show us the way through the Software Dark Ages we're in today: a time that will doubtless be remembered as every bit as mired in darkness and ignorance as the Dark Ages themselves.
But I've failed. This isn't the essay I wanted to write, because I'm neither a great thinker nor a great writer. However, you might be: if not now, then perhaps someday. So I think it's better to get the idea out now than to hoard it in the hopes of someday writing a world-changing essay.
For those of you who were surprised at the suddenness and vehemence of the Lisp community's backlash to my little rant, I hope I've helped shed a little light, helped you see its inevitability. Basically they've had a lot of practice. Lisp is one of the oldest technology religions, and they've both experienced and doled out their share of religious persecution.
But that's not the lesson you should take away. The lesson is that they are you. Whenever you hear someone ranting about something you take for granted as wonderful and praiseworthy, and you're wondering why they don't leave well enough alone so we can all get back to our incestuous cheerleading, just remember: we went from the Dark Ages to our reeeeasonably enlightened society today by questioning our most cherished beliefs.
So keep questioning them.
[1] Yes, I've read all of R6RS. It's a lukewarm compromise that punts on most of the important issues. It's not going to make Scheme any more successful than it is today, which to me feels practically criminal; it was their one big chance to break out of the rut they're in. But it doesn't matter. Let's pretend this footnote is just a troll. If your hackles went up, then you're a techno-religious zombie, and I hope in my lifetime to find you a cure. Try your best to think about that long and hard before responding.
[2] For the record, the commenter I agree the most with is the one who said the problem basically boils down to an IDE issue. SLIME doesn't cut it, either, as beautiful as SLIME is. Can't use it on Windows to save your life, for instance. But that's one of a thousand problems with the Lisp IDE situation; it's pointless to try to discuss them all in blogger. It's probably pointless to discuss them at all, because it's just going to make me more miserable that no decent IDE exists for Lisp, except for Emacs-as-Elisp-IDE. Which is why I get my Lisp fix by hacking elisp these days.
This thought has been nagging at me for a year now, and recently it's been growing like a tumor. One that plenty of folks on the 'net would love to see kill me.
People don't put much stock in philosophers these days. The popular impression of philosophy is that it's just rhetoric, just frivolous debating about stuff that can never properly be answered. "Spare me the philosophy; let's stick to the facts!"
The funny thing is, it's philosophers who gave us the ability to think rationally, to stick to the facts. If it weren't for the work of countless philosophers, facts would still be getting people tortured and killed for discovering and sharing them.
Does it ever strike you as just a teeny bit odd that after a brief period where philosophy flourished, from maybe 400 B.C.E. to ~100 C.E., we went through a follow-on period of well over one thousand five hundred years during which the Roman Catholic Church enslaved everyone's minds and killed anyone who dared think differently?
What's weirder is that we tend to pretend it didn't really happen. We like to just skip right over the dominance of religion over our minds for a hundred generations, and think of religion today as a kindly old grandpa who's just looking out for us kids. No harm, no foul. Let bygones be bygones. Sure, there were massacres and crusades and genocides and torture chambers with teeth grinding and eyes bleeding and intestines torn out in the name of God. But we were all just kids then, right? Nobody does that kind of thing today, at least not in civilized countries.
We try not to think about the uncivilized ones.
It was philosophers that got us out of that Dark Ages mess, and no small number of them lost their lives in doing so. And today, the philosophy majors are the butts of the most jokes, because after the philosophers succeeded in opening our minds, we forgot why we needed them.
And if we stop to think about it at all, we think that it was other people, people who are very unlike us, who committed those atrocities in the name of Faith (regardless of whether it's faith in a god, or in a political party, or any other form of mind control carried out by force).
We like to think we live in an enlightened age, but we don't. Humans haven't changed significantly in 10,000 years. We're still killing and torturing each other. It's apparently incredibly easy to decide to kill someone and then do it. Happens every day, all around the world. Torture, too.
But those people are just people. If they had been born down the street from you, they'd have gone to school with you, been friends with you, learned to program with you, written blogs and comments, never tortured or killed anyone in the name of an idea. They'd have been you. Which means they are you; you just got lucky in where you were born.
One of the commenters on my last blog entry expressed the fervent wish that I drop dead. To be sure, they qualified it with "on the internet". But if they really feel that way, especially about something as hilariously and absurdly unimportant in the Grand Scheme as whether the Lisp programming language has any acceptable implementations, then what does it say about us?
Everyone who commented angrily on that blog entry was caught. I caught you, anonymous or not, being a religious fanatic. The only "negative" commenter who doesn't appear to be a religious zombie was Paul Costanza (ironic, since he claims to be the opinionated one), who relegated his comments to pedantic technical corrections. They're welcome, of course; I'm always looking to correct any technical misconceptions I harbor. But they're moot, since even if I was wrong about every single technical point I brought up in that entry, my overall point — Lisp is not an acceptable Lisp — remains largely uncontested by the commenters.
Some of them just don't get it, which is fine; no harm in that. If you've been using Lisp for years and years, and you've written books and articles and zillions of lines of Lisp code, then you're unlikely to remember anything about what it's like coming to Lisp for the first time. They're religious because they've forgotten what it's like to be a skeptic.
But make no mistake; a substantial percentage of people who take a side in any programming language discussion that devolves into a flamewar know exactly what the other side means, and they want to invoke the Ultimate Censorship: drop dead! Killing someone, after all, is one of the best ways to silence them. You also have to burn all their writings, which is getting harder these days; hence the increased vehemence on the 'net.
Those of you who've followed what I've written over the past year or so know where I'm going. I'm taking a stand, all right, and it's a very definite one. I'm finding myself drawn inexorably towards a single goal: stamping out technological religion, because I'm frigging tired of not being able to stick to the facts.
FACT: Java has no first-class functions and no macros. This results in warped code that hacks around the problem, and as the code base grows, it takes on a definite, ugly shape, one that's utterly unique to Java. Lisp people can see this clear as day. So can Python folks, so can Ruby folks. Java people flip out, and say "macros are too much power", or "what do u mean i dont understand u" or "fuck you, you jerk, Lisp will NEVER win".
You think I don't hear ALL that, and much more, in the hate mail I get every day?
I sure wouldn't want to be alone with a Java fanatic in a medieval torture chamber, because God only knows what they're capable of.
Turn the mirror towards Python, and what happens? Funny, but the Java folks will mail me saying: "yeah, I've always known I detested Python, and you really nailed exactly why. Thanks!" Meanwhile, Python folks are literally frothing at the mouth, looking for the "Kill That Bastard" key on their 101-key keyboards.
I turned the mirror towards Lisp yesterday. Had to go to the bathroom like nobody's business, and my wife was expecting me home any minute, so I rushed it out: just a few thoughts here and there. So the Gorgon only caught the tiniest glimpse of itself, but hell evidently hath no fury like that of a Lisper scorned, and all that.
It doesn't matter that I rushed it out. I'm glad I did; spending any more time on it, trying to get it "right" by looking up useless factoids like how you can override length's non-polymorphicness with some weird setting (when it plainly should just be the default), would have had the exact same net effect: Lisp zealots would have found some way to turn it into a flamewar. And I'd have been out 2 or 3 more hours.
Let's call it a troll, then, because it was poorly researched; it was just some months-old recollections of pain I'd gone through last year trying to commit to Common Lisp, after another year of trying the same with various flavors of Scheme and finding them all wanting. As far as I'm concerned, Lisp is unacceptable today; it's my opinion and just that, but I'll stick with it.
I still need Lisp; after you learn enough of it, it becomes part of your soul. I get my fix hacking elisp, and I do a lot of it. The commenters are quite right; I've never written anything substantial in Common Lisp, because in each of my serious attempts, there was too much friction. Risk/reward wasn't high enough, and believe me, I wanted it.
But after many attempts, I've given up on Common Lisp. They won't let me use it where I work, and there are probably more Lispers per capita where I work, including some famous ones, than at any other big company in the world. If we can't use it where I work, then it's frigging unacceptable; that's the shortest proof I can offer.
What I'm far more interested today is the situation that arises if you consider my post a troll. I'm far more interested in the social consequences of working in a world filled with religious fanatics of different religious persuasions. Especially given that it's a world in which "natural religion" has, by and large, been marginalized through the work of philosophers.
Let's look at this world in a little more detail, starting with Peter Siebel's comment, which I believe is the most interesting. Peter said:
I was trying to figure out why on earth you spent so much time writing about something that you apparently don't like. Then it hit me: HCGS. So thanks for your help.
His first sentence speaks volumes about the sociology. His viewpoint is exactly what they teach us all as kids: If you don't have anything nice to say, don't say anything at all. We like to think people have a right to believe whatever they want, and that it's not nice to say mean things about other people's beliefs, especially when their livelihoods are at stake.
That's where philosophers come in, folks. They pick your beliefs apart and show you in unforgettable ways the consequences of what you believe in. I'm no philosopher; I know basically nothing about it, but I can tell you I wish fervently that some great philosophers would come along and effect change in our technical society.
Because if nothing else, I can see the consequences of the way we're thinking about things. One of many such consequences is that languages aren't getting any better, and the worst offenders are Lisp and Scheme, which by rights should be racing along the innovation curve faster than their supposedly less capable peers. But they've stagnated worse than any other non-dead language I can think of.[1]
Programming languages are religions. For a long while now I've been mildly uncomfortable calling it "religion", but I don't feel bad about it anymore. They're similar enough. At the top of the language religion is the language itself; it serves as the deity and the object of worship.
Like any other organized religion, there's always a Pope (or a politburo chairman, in countries where the government has brutally set itself up as what is for all intents the religion of choice): a spiritual leader that gives the religion the human touch. This person is almost always the language designer, of course. In Lisp's case it's complicated, because McCarthy, Sussman and Steele aren't very active as spiritual leaders for their languages anymore.
Every major organized religion is a heirarchical government, and programming languages are no exception. You'll find equivalents of cardinals, bishops, priests and laity in programming language camps: the closer you are to the fire, to the spiritual center, the higher your rank. It's a great way to quantify your perceived self-importance: a high-score list, in effect. Great for the ego, but it makes you a piss-poor debater, because you're so emotionally invested in your status.
You'd think your rank would be accrued by virtue of your technical and/or documentation contributions, but in practice it's usually more of a function of how many converts you've gained, how many followers you have, how much you've been spreading the Word.
That's why Paul Graham isn't the Pope of Lisp. He's eminently qualified, but unfortunately he's a heretic. Notice that almost none of the commenters on my last blog mentioned the PG argument I made. The only one who did (as of this writing) tried to make it an argument for Common Lisp. Let's face it: you can't give those heretics too much press; people might start listening to them!
Peter, are you beginning to understand why I write so much about something I apparently don't like? It's because I wanted to like it but found it fatally flawed, technically and culturally. It's as if I were a would-be convert to Roman Catholicism, but I can't bring myself to commit because I've seen too much of their role in creating a history that ironically we all wish we could rewrite.
I was born and raised a Roman Catholic, and I renounced it when I was thirteen years old, after my Uncle Frank (a devout terrorist Catholic if there ever was one) told me to stop reading the Bible, that it would "really screw a person up" to do that, that you needed someone to interpret it for you. That wasn't the only reason I renounced it, but it'll suffice for our purposes.
Technologically I was born and raised an assembly-language programmer; at least that's what my first real job was, for 5 years after I got my CS degree. Assembly is just flagellation, though, and damned uncomfortable at that, so I joined the Church of Java for fully seven years. And practically at the very moment I'd finally tired of chafing at Java's limitations, Paul Graham came along and through his early essays, showed me Lisp. What a great new religion!
Problem is, each time you switch religions, the next one has less impact on you. Once a Catholic, always a Catholic, they say. I don't know what that means for me, since I was raised by the assembly-language wolf, but it appears to mean that I'm never going to be enthralled with another programming language. And now that I've swallowed the red pill, what choice do I have? I need to try to show people what's out there.
Interestingly, it was Peter Siebel's most excellent book, Practical Common Lisp, that played the role of Uncle Frank and killed my desired to continue with Common Lisp. Peter was the first person to show me beast's underbelly. Every other Lisp book had pretended it was pure and beautiful and uncorrupted, because they left all the nastiness out as "implementation-defined". Once I saw what you really need to do in order to build something resembling a portable Lisp code base, and then had a few runs at it myself, I threw in the towel.
I much prefer Lisp the idea to Lisp the implementation.[2]
I can tell you this: I've tried writing this essay for a year. I've tried fully a dozen times. I've tackled it from a dozen angles. I've wanted to say it — software needs philosophers! — so many times, in so many ways. We need great thinkers — the Fyodor Dostoyevskys and David Humes and Aristotles and Jean-Paul Sartres and Ben Franklins and Galileo Galileis and Bertrand Russells and Albert Einsteins to show us the way through the Software Dark Ages we're in today: a time that will doubtless be remembered as every bit as mired in darkness and ignorance as the Dark Ages themselves.
But I've failed. This isn't the essay I wanted to write, because I'm neither a great thinker nor a great writer. However, you might be: if not now, then perhaps someday. So I think it's better to get the idea out now than to hoard it in the hopes of someday writing a world-changing essay.
For those of you who were surprised at the suddenness and vehemence of the Lisp community's backlash to my little rant, I hope I've helped shed a little light, helped you see its inevitability. Basically they've had a lot of practice. Lisp is one of the oldest technology religions, and they've both experienced and doled out their share of religious persecution.
But that's not the lesson you should take away. The lesson is that they are you. Whenever you hear someone ranting about something you take for granted as wonderful and praiseworthy, and you're wondering why they don't leave well enough alone so we can all get back to our incestuous cheerleading, just remember: we went from the Dark Ages to our reeeeasonably enlightened society today by questioning our most cherished beliefs.
So keep questioning them.
[1] Yes, I've read all of R6RS. It's a lukewarm compromise that punts on most of the important issues. It's not going to make Scheme any more successful than it is today, which to me feels practically criminal; it was their one big chance to break out of the rut they're in. But it doesn't matter. Let's pretend this footnote is just a troll. If your hackles went up, then you're a techno-religious zombie, and I hope in my lifetime to find you a cure. Try your best to think about that long and hard before responding.
[2] For the record, the commenter I agree the most with is the one who said the problem basically boils down to an IDE issue. SLIME doesn't cut it, either, as beautiful as SLIME is. Can't use it on Windows to save your life, for instance. But that's one of a thousand problems with the Lisp IDE situation; it's pointless to try to discuss them all in blogger. It's probably pointless to discuss them at all, because it's just going to make me more miserable that no decent IDE exists for Lisp, except for Emacs-as-Elisp-IDE. Which is why I get my Lisp fix by hacking elisp these days.
Jumat, 14 April 2006
Lisp is Not an Acceptable Lisp
It's been over four months since Eric Kidd posted his infamous Why Ruby is an acceptable LISP article. You know, the one that got approximately 6.02e23 comments, ranging from "I agree!" through "I hate you!" to "I bred them together to create a monster!" Any time the comment thread becomes huge enough to exhibit emergent behavior, up to and including spawning new species of monsters, you know you've touched a nerve.
What amazes me is that nobody's pointed out the obvious counter-observation: Lisp is not an acceptable LISP. Not for any value of Lisp. There's nothing magical about this, nothing partisan. If Lisp were acceptable, then we'd all be using it.
You've all read about the Road to Lisp. I was on it for a little over a year. It's a great road, very enlightening, blah blah blah, but what they fail to mention is that Lisp isn't the at the end of it. Lisp is just the last semi-civilized outpost you hit before it turns into a dirt road, one that leads into the godawful swamp most of us spend our programming careers slugging around in. I guarantee you there isn't one single Lisp programmer out there who uses exclusively Lisp. Instead we spend our time hacking around its inadequacies, often in other languages.
So for four months I've been waiting for someone else to say it, but so far it's not happening. Why aren't we admitting it?
Oh, right. Religion. I keep forgetting about that.
Lisp programmers are just like all other programmers: they want to write code and get cool stuff done, which presupposes they've already learned the last programming language they'll ever need. There's all this real-life stuff (jobs, family, stability, all the usual suspects) intruding on you as a programmer, demanding that you quit dorking around looking for the One True Language, and settle down on whatever barren rock you happen to be squatting on at the moment, and call it Good. So most Lisp programmers — and that's not many, since not many programmers make it even close to that far down the Road — see that last outpost of technical civilization, peer balefully into the swamp, and decide to check into the Lisp hotel for good. Not realizing, of course, that all its rooms are in the swamp proper.
We all know it's tricky to have a rational discussion about a religion. Non-Lispers will be able to read this without getting their feathers ruffled. Some Lispers aren't too far gone, so let's assume we're talking to them, and take a look at some of Lisp's problems that make it flat-out unacceptable. At least for LISP, you know, the idealized one.
Problem 1: Which Lisp?
Sorry, folks, but you can't trivialize this one. Let's say I'm a new would-be Lisper, just finished walking down that long damn Road, and now that I'm here, I'm ready to start using it. Which "it" should I use? The answer is "it depends", and that's pretty unfortunate, because right there you've just lost users. With Python or Ruby or Java, you've only got one language to choose from. Or at least you can be comfortable that there's a single canonical version, and the rest (e.g. Jython) are highly experimental territory.
Pick Scheme, and you have to pick a Scheme. Pick Common Lisp, and you have to pick a Common Lisp. Heck, there are even two or three flavors of Emacs-Lisp out there.
Most newcomers eventually (and independently) decide the same thing: Scheme is a better language, but Common Lisp is the right choice for production work. CL has more libraries, and the implementations are somewhat more compatible than Scheme implementations, particularly with respect to macros. So newcomers heave a deep sigh, and they learn to accept LISP-2, names like rplaca, case-insensitivity, '(ALL CAPS OUTPUT), and all the other zillions of idiosyncracies of a standard Common Lisp implementation. Eventually, if they stick with Lisp at all, they learn they can override most of these defaults in nonportable ways, which makes things infinitesimally more bearable.
Whatever. If you're a Lisper, you dealt with all this crap years ago, and now you're committed. If you're not a Lisper, then you're not very likely to become one any time soon. In fact your probability of learning Lisp is decreasing over time, as other languages continue to close the gap in the Lispy areas, and simultaneously increase their lead in non-Lispy areas where Lisp is making little (if any) progress.
Let's look at some of those areas. But first, let me mention one last problem in the "which Lisp" space. It's dirty laundry that needs airing. The problem: Paul Graham. I mean, the guy's a genius, and I love reading his essays, and his startups are doing great things, etc. etc. You can't fault him. But he's created something of a problem.
Before Paul Graham, Lisp was dying. It really was, and let's not get all sentimental or anything; it's just common sense. A language is always either gaining or losing ground, and Lisp was losing ground all through the 1990s. Then PG came along with his "I'm not talking to you if you're over 26 years old" essays, each a giant slap in our collective face, and everyone sat up and paid attention to him in a hurry. And a TON of people started looking very seriously at Lisp.
Lisp might or might not have experienced a revival without Paul's essays, but it's moot: he showed up, and Lisp got real popular, real fast. And then he said: "Don't use it!" Sort of. I mean, that's effectively what he said, isn't it? By deciding to pre-announce Arc, he Microsofted Lisp. Killed it with vaporware. It's a great strategy when you're an evil empire. I don't think that's exactly what Paul had in mind, but let's face it: that's what happened.
So Common Lispers grumble about Paul in the hallways. If I read newsgroups (every time I try, the overall ugliness of humanity drives me away within hours; I only re-attempt it every decade or so) I see them grumbling there too. He's put them in a tough spot, because he did use Common Lisp for Viaweb, and his arguments in favor of Lisp (in the general sense) have been compelling enough to bring in newcomers by the droves. But he's not throwing his weight behind CL. He's not even taking the marginally-acceptable route (from a damage-control perspective) of recommending Scheme. Instead, Paul's (*gasp*) starting a new religion.
Arc's going to be a new religion, of course, because programmers just haaaaaave to make it that way. If it ever appears, anyway. But will it? That's the tricky thing about Cathedral-style software; you never can tell. My prediction: someone will get tired of waiting, and they'll Torvalds Arc into obsolescence before it's ever released. (If you don't get the reference, it's what Linux did to GNU Hurd).
Long story short: nobody knows what the hell Lisp they're supposed to be using, and it's absolutely killing adoption.
Problem 2: Worthless Spec
Oh, ouch, did I have to put it quite like that? I mean, c'mon, let's be fair, there are literally hundreds of people out there who disagree.
Unfortunately, the simple fact is that the spec is ancient. Every time someone talks about updating it, someone screams about time or money or whatever. The problem is (like the problem with RSS) a people-problem, not a time or money problem. This is absolutely true in the Scheme world, too. There are a bunch of old-timer stakeholders who want to have their say. So you're basically asking a goverment (complete with lobbyists, political parties, the works) to design Lisp if you go that route. The naysayers are right about one thing: it'll never happen.
Your only other option is to design a new language, and you won't get any help from Lisp people, because they will hate you. They love pointing to the trail of bodies left in the wake of every pioneer who's tried this before, none of whom has emerged with a "successful" Lisp. Of course, they haven't been successful because Lispers didn't want to have anything to do with them; Lispers are just as incapacitated by their techno-religious beliefs as folks from other languages. Religions dislike each other, but no heretic is as damned as someone who starts with your religion and makes a modification to it. Just ask the Albigensians, for instance.
But what's wrong with Common Lisp? Do I really need to say it? Every single non-standard extension, everything not in the spec, is "wrong" with Common Lisp. This includes any support for threads, filesystem access, processes and IPC, operating system interoperability, a GUI, Unicode, and the long list of other features missing from the latest hyperspec.
Effectively, everything that can't be solved from within Lisp is a target. Lisp is really powerful, sure, but some features can only be effective if they're handled by the implementation.
Problem 3: CLOS
Note: this section contains a few factual errors pointed out by Pascal Costanza in a comment below. Some of his corrections are also, of course, opinions, and I've commented on them later in the thread. In any case, while I thank Pascal for his corrections, the errors I've made are utterly irrelevant to my conclusions.
CLOS is icky. I haven't worked with Smalltalk a whole lot, but I've worked with it enough to know that to do OOP right, you have to do it from the ground up. CLOS was bolted on to Common Lisp. Everyone knows it, although not many people want to admit it.
It was bolted on very nicely, and it's not my intention to disparage the efforts of the people who created it. It was an amazing piece of work, and it did a great job of being flexible enough to tie together the conflicting OO systems of existing Lisp implementations.
But let's face it; CLOS has problems. One obvious one is that
Another problem is the slot accessor macros. They're insanely clever, but clever isn't what you want. You want first-class function access, so you can pass the getters and setters to
What you really want in lieu of CLOS is... complicated. It's a hard problem. Lisp wants to be constructed entirely from macros. It's part of the purity of the idea of LISP: you only need the seven (or is it five?) primitives to build the full machine. Doing CLOS as a bunch of macros was very much in the spirit of Lisp: it was a Lispy thing to do.
But macros are a problem. Yes, they're one of the most important differentiators. But macros are like having these high-powered band-aids, when what you want is not to be wounded in the first place. Having the object system — something pretty fundamental to the language, you'd think — written as a bunch of macros doesn't feel right when all is said and done.
When you work with Ruby or Smalltalk or any suitably "pure" OO language (Python doesn't quite count, unfortunately; its bolts are also showing), you realize there are some distinct advantages to having everything be an object. It's very nice, for instance, to be able to figure out what methods are applicable to a given class (e.g.
Of course, that forces you into a single-dispatch model, so it becomes harder to figure out what to do about multi-methods. Some Python folks have implemented multi-methods for Python, and they do it by making them top-level functions, which makes sense (where else would you put them?) I'm not claiming that Smalltalk's object model is going to translate straight to Lisp; you have to decide whether cons cells are "objects", for instance, and that's a decision I wouldn't wish on my worst enemy. I don't envy the person who tackles it.
Regardless of what the solution might be, CLOS remains a problem. It's over-complicated and yet not quite OOP-y enough or expressive enough. The problem of reflecting on an object to see which methods are valid for it is one example, but there are tons of others. Heck, one possibly valid complaint is that it doesn't work very much like the "conventional" OOP systems of C++, Java, Python and Ruby. There's no real reason it shouldn't be more like them. But changing CLOS to be simpler and more seamless essentially means replacing it. And replacing it is probably best done inside the implementation.
In other words, any fix means starting virtually from scratch.
Or maybe you could go the Haskell route and not have OOP at all. That seems to alienate most programmers, though, despite the attractions of not having to create nouns for everything. (Have you ever noticed that turning a non-object-oriented program into an object-oriented one in the same language that does the same thing essentially doubles its size? Try it sometime...) At the risk of predicting future fashion trends, which is rarely a good idea, I'll venture that objects are going to continue to be trendy for at least a few more decades. So I think Lisp needs some form of "seamless" OOP.
Problem 4: Macros
Macros are one of the worst problems with Lisp, or at least they're one of the biggest unsolved problems.
Yes, they're amazingly powerful and critically important and blah Blah BLAH. You can read all about them elsewhere. Paul Graham's On Lisp is the best reference I've found.
But they're fraught with problems. One is that they're not hygienic. You should at least have the option of requesting hygienic macros. Various papers have been published, and implementations implemented, for hygienic
For that matter, you should also have a choice between Scheme-style pattern-matching macros and Lisp-style code-style macros. They're very different, and each kind is better (cleaner) in some situations. People often act as if hygiene is synonymous with define-syntax, but the pattern-template style is orthogonal to the question of hygiene.
Style considerations aside, macros have tool problems. Macros are notoriously hard to debug, and honestly it needn't be that way. If your editor knows all about macros, then you should be able to click to see the expansion, and click again to see its sub-expansions, all the way down to the primitive functions. Some editors can do this, but none of them (that I'm aware of) handle macros as cleanly or seamlessly as they do normal functions.
Syntax in general is a problem. Lisp has a little syntax, and it shows up occasionally as, for instance, '(foo) being expanded as (quote foo), usually when you least expect it. Truth be told, Lisp should probably have a skinnable syntax. That implies a canonical abstract syntax tree, which of course hasn't been defined (and in many implementations isn't even available to you, the way it is in the Io language, say). Once you've got a canonical AST defined, syntax should, in theory, be like CSS chrome. Of course, there are plenty of bodies left in the trail of this particular theory as well. Someday...
In any case, because macros are rarely supported "well enough" by the tools, and because they're not first-class functions, and so on, they wind up being second-class citizens. The rule "you should only use a macro when nothing else will do" implies that they really are a last resort, which (to me) is synonymous with band-aid. Yes, it's wonderful that you have the band-aid — or maybe duct tape is a better analogy — certainly you miss them dearly when you're working in other languages. But you don't want to have to build your entire object system with duct tape.
So macros, like the object system, need to be re-thought from the ground up. There's undoubtedly enough research in the space that someone could throw together a working definition in no time, something just good enough for today's programmers, the ones who expect (and rightfully so, I might add) to be able to name their methods "length" without getting a compiler error.
Problem 4: Type System
See, that's just exactly the problem with type systems. They can make sure you use headings, but they can't ensure you get the numbering right.
Well, it'll take me forever to talk about this one, so I'll have to leave it for another blog. The problem is that the type system has to be extensible and skinnable, and I'm not strictly talking about user-defined types in the sense of OOP or CLOS. Unfortunately it really is a huge open issue, one that'll take longer than this blog to sort through, so I'll have to leave it for today.
Lisp, for all the strengths of its flexible type system, hasn't got this issue right either. Otherwise Haskell and OCaml (and C++, gack) wouldn't be kicking its ass all over the performance map. 'nuff said, at least for now. [And no, they don't quite have it right either.]
I promise I'll talk about type systems soon. But I also promised some friends I'd make my blogs shorter.
There is no acceptable Lisp
This is a problem. It's not a little teeny one, either. The Lisp communities (yeah, there are a bunch) are going to have to realize that if Lisp is ever going to be massively successful, it needs an overhaul. Or maybe a revolution. Contrary to what some might tell you, it doesn't need a committee, and it doesn't need a bunch of money. Linux proved exactly the opposite. Lisp needs a benevolent dictator. Lisp needs to ditch the name "Lisp", since it scares people. And Lisp needs to learn from the lessons of the 45 years of languages that have followed it.
And no, I'm not the guy. You're all far more qualified to tackle this problem than I am. Especially if you're under 26.
What amazes me is that nobody's pointed out the obvious counter-observation: Lisp is not an acceptable LISP. Not for any value of Lisp. There's nothing magical about this, nothing partisan. If Lisp were acceptable, then we'd all be using it.
You've all read about the Road to Lisp. I was on it for a little over a year. It's a great road, very enlightening, blah blah blah, but what they fail to mention is that Lisp isn't the at the end of it. Lisp is just the last semi-civilized outpost you hit before it turns into a dirt road, one that leads into the godawful swamp most of us spend our programming careers slugging around in. I guarantee you there isn't one single Lisp programmer out there who uses exclusively Lisp. Instead we spend our time hacking around its inadequacies, often in other languages.
So for four months I've been waiting for someone else to say it, but so far it's not happening. Why aren't we admitting it?
Oh, right. Religion. I keep forgetting about that.
Lisp programmers are just like all other programmers: they want to write code and get cool stuff done, which presupposes they've already learned the last programming language they'll ever need. There's all this real-life stuff (jobs, family, stability, all the usual suspects) intruding on you as a programmer, demanding that you quit dorking around looking for the One True Language, and settle down on whatever barren rock you happen to be squatting on at the moment, and call it Good. So most Lisp programmers — and that's not many, since not many programmers make it even close to that far down the Road — see that last outpost of technical civilization, peer balefully into the swamp, and decide to check into the Lisp hotel for good. Not realizing, of course, that all its rooms are in the swamp proper.
We all know it's tricky to have a rational discussion about a religion. Non-Lispers will be able to read this without getting their feathers ruffled. Some Lispers aren't too far gone, so let's assume we're talking to them, and take a look at some of Lisp's problems that make it flat-out unacceptable. At least for LISP, you know, the idealized one.
Problem 1: Which Lisp?
Sorry, folks, but you can't trivialize this one. Let's say I'm a new would-be Lisper, just finished walking down that long damn Road, and now that I'm here, I'm ready to start using it. Which "it" should I use? The answer is "it depends", and that's pretty unfortunate, because right there you've just lost users. With Python or Ruby or Java, you've only got one language to choose from. Or at least you can be comfortable that there's a single canonical version, and the rest (e.g. Jython) are highly experimental territory.
Pick Scheme, and you have to pick a Scheme. Pick Common Lisp, and you have to pick a Common Lisp. Heck, there are even two or three flavors of Emacs-Lisp out there.
Most newcomers eventually (and independently) decide the same thing: Scheme is a better language, but Common Lisp is the right choice for production work. CL has more libraries, and the implementations are somewhat more compatible than Scheme implementations, particularly with respect to macros. So newcomers heave a deep sigh, and they learn to accept LISP-2, names like rplaca, case-insensitivity, '(ALL CAPS OUTPUT), and all the other zillions of idiosyncracies of a standard Common Lisp implementation. Eventually, if they stick with Lisp at all, they learn they can override most of these defaults in nonportable ways, which makes things infinitesimally more bearable.
Whatever. If you're a Lisper, you dealt with all this crap years ago, and now you're committed. If you're not a Lisper, then you're not very likely to become one any time soon. In fact your probability of learning Lisp is decreasing over time, as other languages continue to close the gap in the Lispy areas, and simultaneously increase their lead in non-Lispy areas where Lisp is making little (if any) progress.
Let's look at some of those areas. But first, let me mention one last problem in the "which Lisp" space. It's dirty laundry that needs airing. The problem: Paul Graham. I mean, the guy's a genius, and I love reading his essays, and his startups are doing great things, etc. etc. You can't fault him. But he's created something of a problem.
Before Paul Graham, Lisp was dying. It really was, and let's not get all sentimental or anything; it's just common sense. A language is always either gaining or losing ground, and Lisp was losing ground all through the 1990s. Then PG came along with his "I'm not talking to you if you're over 26 years old" essays, each a giant slap in our collective face, and everyone sat up and paid attention to him in a hurry. And a TON of people started looking very seriously at Lisp.
Lisp might or might not have experienced a revival without Paul's essays, but it's moot: he showed up, and Lisp got real popular, real fast. And then he said: "Don't use it!" Sort of. I mean, that's effectively what he said, isn't it? By deciding to pre-announce Arc, he Microsofted Lisp. Killed it with vaporware. It's a great strategy when you're an evil empire. I don't think that's exactly what Paul had in mind, but let's face it: that's what happened.
So Common Lispers grumble about Paul in the hallways. If I read newsgroups (every time I try, the overall ugliness of humanity drives me away within hours; I only re-attempt it every decade or so) I see them grumbling there too. He's put them in a tough spot, because he did use Common Lisp for Viaweb, and his arguments in favor of Lisp (in the general sense) have been compelling enough to bring in newcomers by the droves. But he's not throwing his weight behind CL. He's not even taking the marginally-acceptable route (from a damage-control perspective) of recommending Scheme. Instead, Paul's (*gasp*) starting a new religion.
Arc's going to be a new religion, of course, because programmers just haaaaaave to make it that way. If it ever appears, anyway. But will it? That's the tricky thing about Cathedral-style software; you never can tell. My prediction: someone will get tired of waiting, and they'll Torvalds Arc into obsolescence before it's ever released. (If you don't get the reference, it's what Linux did to GNU Hurd).
Long story short: nobody knows what the hell Lisp they're supposed to be using, and it's absolutely killing adoption.
Problem 2: Worthless Spec
Oh, ouch, did I have to put it quite like that? I mean, c'mon, let's be fair, there are literally hundreds of people out there who disagree.
Unfortunately, the simple fact is that the spec is ancient. Every time someone talks about updating it, someone screams about time or money or whatever. The problem is (like the problem with RSS) a people-problem, not a time or money problem. This is absolutely true in the Scheme world, too. There are a bunch of old-timer stakeholders who want to have their say. So you're basically asking a goverment (complete with lobbyists, political parties, the works) to design Lisp if you go that route. The naysayers are right about one thing: it'll never happen.
Your only other option is to design a new language, and you won't get any help from Lisp people, because they will hate you. They love pointing to the trail of bodies left in the wake of every pioneer who's tried this before, none of whom has emerged with a "successful" Lisp. Of course, they haven't been successful because Lispers didn't want to have anything to do with them; Lispers are just as incapacitated by their techno-religious beliefs as folks from other languages. Religions dislike each other, but no heretic is as damned as someone who starts with your religion and makes a modification to it. Just ask the Albigensians, for instance.
But what's wrong with Common Lisp? Do I really need to say it? Every single non-standard extension, everything not in the spec, is "wrong" with Common Lisp. This includes any support for threads, filesystem access, processes and IPC, operating system interoperability, a GUI, Unicode, and the long list of other features missing from the latest hyperspec.
Effectively, everything that can't be solved from within Lisp is a target. Lisp is really powerful, sure, but some features can only be effective if they're handled by the implementation.
Problem 3: CLOS
Note: this section contains a few factual errors pointed out by Pascal Costanza in a comment below. Some of his corrections are also, of course, opinions, and I've commented on them later in the thread. In any case, while I thank Pascal for his corrections, the errors I've made are utterly irrelevant to my conclusions.
CLOS is icky. I haven't worked with Smalltalk a whole lot, but I've worked with it enough to know that to do OOP right, you have to do it from the ground up. CLOS was bolted on to Common Lisp. Everyone knows it, although not many people want to admit it.
It was bolted on very nicely, and it's not my intention to disparage the efforts of the people who created it. It was an amazing piece of work, and it did a great job of being flexible enough to tie together the conflicting OO systems of existing Lisp implementations.
But let's face it; CLOS has problems. One obvious one is that
length isn't a polymorphic function. It's one of the first speed bumps you encounter. You can't create a new kind of measurable object and give it a length method; you have to call it rope-length or foo-length or whatever. That's part of Lisp's endoskeleton showing; you can see the bolt sticking out plain as the nose on your face. It's not seamless; it's not orthogonal, and it's not the Right Thing. But it's not going to change, either.Another problem is the slot accessor macros. They're insanely clever, but clever isn't what you want. You want first-class function access, so you can pass the getters and setters to
map, find-if, etc. You can work around these things, but they're a leaky abstraction, and enough of those will add up to significant mental resistance to getting things done. It's like all those weird little rules in Perl: non-orthogonal rules that add up to forgetting the language every time you leave it alone for more than a week.What you really want in lieu of CLOS is... complicated. It's a hard problem. Lisp wants to be constructed entirely from macros. It's part of the purity of the idea of LISP: you only need the seven (or is it five?) primitives to build the full machine. Doing CLOS as a bunch of macros was very much in the spirit of Lisp: it was a Lispy thing to do.
But macros are a problem. Yes, they're one of the most important differentiators. But macros are like having these high-powered band-aids, when what you want is not to be wounded in the first place. Having the object system — something pretty fundamental to the language, you'd think — written as a bunch of macros doesn't feel right when all is said and done.
When you work with Ruby or Smalltalk or any suitably "pure" OO language (Python doesn't quite count, unfortunately; its bolts are also showing), you realize there are some distinct advantages to having everything be an object. It's very nice, for instance, to be able to figure out what methods are applicable to a given class (e.g.
"foo".methods.sort.grep(/!/) from Ruby), and to be able to extend that list with your own new methods. It's a nice organizational technique.Of course, that forces you into a single-dispatch model, so it becomes harder to figure out what to do about multi-methods. Some Python folks have implemented multi-methods for Python, and they do it by making them top-level functions, which makes sense (where else would you put them?) I'm not claiming that Smalltalk's object model is going to translate straight to Lisp; you have to decide whether cons cells are "objects", for instance, and that's a decision I wouldn't wish on my worst enemy. I don't envy the person who tackles it.
Regardless of what the solution might be, CLOS remains a problem. It's over-complicated and yet not quite OOP-y enough or expressive enough. The problem of reflecting on an object to see which methods are valid for it is one example, but there are tons of others. Heck, one possibly valid complaint is that it doesn't work very much like the "conventional" OOP systems of C++, Java, Python and Ruby. There's no real reason it shouldn't be more like them. But changing CLOS to be simpler and more seamless essentially means replacing it. And replacing it is probably best done inside the implementation.
In other words, any fix means starting virtually from scratch.
Or maybe you could go the Haskell route and not have OOP at all. That seems to alienate most programmers, though, despite the attractions of not having to create nouns for everything. (Have you ever noticed that turning a non-object-oriented program into an object-oriented one in the same language that does the same thing essentially doubles its size? Try it sometime...) At the risk of predicting future fashion trends, which is rarely a good idea, I'll venture that objects are going to continue to be trendy for at least a few more decades. So I think Lisp needs some form of "seamless" OOP.
Problem 4: Macros
Macros are one of the worst problems with Lisp, or at least they're one of the biggest unsolved problems.
Yes, they're amazingly powerful and critically important and blah Blah BLAH. You can read all about them elsewhere. Paul Graham's On Lisp is the best reference I've found.
But they're fraught with problems. One is that they're not hygienic. You should at least have the option of requesting hygienic macros. Various papers have been published, and implementations implemented, for hygienic
defmacro. Yeah, it's hellishly hard to get right, and it's overkill for many situations, but it really does need to be offered as an option. A portable one.For that matter, you should also have a choice between Scheme-style pattern-matching macros and Lisp-style code-style macros. They're very different, and each kind is better (cleaner) in some situations. People often act as if hygiene is synonymous with define-syntax, but the pattern-template style is orthogonal to the question of hygiene.
Style considerations aside, macros have tool problems. Macros are notoriously hard to debug, and honestly it needn't be that way. If your editor knows all about macros, then you should be able to click to see the expansion, and click again to see its sub-expansions, all the way down to the primitive functions. Some editors can do this, but none of them (that I'm aware of) handle macros as cleanly or seamlessly as they do normal functions.
Syntax in general is a problem. Lisp has a little syntax, and it shows up occasionally as, for instance, '(foo) being expanded as (quote foo), usually when you least expect it. Truth be told, Lisp should probably have a skinnable syntax. That implies a canonical abstract syntax tree, which of course hasn't been defined (and in many implementations isn't even available to you, the way it is in the Io language, say). Once you've got a canonical AST defined, syntax should, in theory, be like CSS chrome. Of course, there are plenty of bodies left in the trail of this particular theory as well. Someday...
In any case, because macros are rarely supported "well enough" by the tools, and because they're not first-class functions, and so on, they wind up being second-class citizens. The rule "you should only use a macro when nothing else will do" implies that they really are a last resort, which (to me) is synonymous with band-aid. Yes, it's wonderful that you have the band-aid — or maybe duct tape is a better analogy — certainly you miss them dearly when you're working in other languages. But you don't want to have to build your entire object system with duct tape.
So macros, like the object system, need to be re-thought from the ground up. There's undoubtedly enough research in the space that someone could throw together a working definition in no time, something just good enough for today's programmers, the ones who expect (and rightfully so, I might add) to be able to name their methods "length" without getting a compiler error.
Problem 4: Type System
See, that's just exactly the problem with type systems. They can make sure you use headings, but they can't ensure you get the numbering right.
Well, it'll take me forever to talk about this one, so I'll have to leave it for another blog. The problem is that the type system has to be extensible and skinnable, and I'm not strictly talking about user-defined types in the sense of OOP or CLOS. Unfortunately it really is a huge open issue, one that'll take longer than this blog to sort through, so I'll have to leave it for today.
Lisp, for all the strengths of its flexible type system, hasn't got this issue right either. Otherwise Haskell and OCaml (and C++, gack) wouldn't be kicking its ass all over the performance map. 'nuff said, at least for now. [And no, they don't quite have it right either.]
I promise I'll talk about type systems soon. But I also promised some friends I'd make my blogs shorter.
There is no acceptable Lisp
This is a problem. It's not a little teeny one, either. The Lisp communities (yeah, there are a bunch) are going to have to realize that if Lisp is ever going to be massively successful, it needs an overhaul. Or maybe a revolution. Contrary to what some might tell you, it doesn't need a committee, and it doesn't need a bunch of money. Linux proved exactly the opposite. Lisp needs a benevolent dictator. Lisp needs to ditch the name "Lisp", since it scares people. And Lisp needs to learn from the lessons of the 45 years of languages that have followed it.
And no, I'm not the guy. You're all far more qualified to tackle this problem than I am. Especially if you're under 26.
Kamis, 30 Maret 2006
Execution in the Kingdom of Nouns
| They've a temper, some of them—particularly verbs: they're the proudest—adjectives you can do anything with, but not verbs—however, I can manage the whole lot of them! Impenetrability! That's what I say! — Humpty Dumpty |
Hello, world! Today we're going to hear the story of Evil King Java and his quest for worldwide verb stamp-outage.1
Caution: This story does not have a happy ending. It is neither a story for the faint of heart nor for the critical of mouth. If you're easily offended, or prone to being a disagreeable knave in blog comments, please stop reading now.
Before we begin the story, let's get some conceptual gunk out of the way.
The Garbage Overfloweth
All Java people love "use cases", so let's begin with a use case: namely, taking out the garbage. As in, "Johnny, take out that garbage! It's overflowing!"
If you're a normal, everyday, garden-variety, English-speaking person, and you're asked to describe the act of taking out the garbage, you probably think about it roughly along these lines:
get the garbage bag from under the sink
carry it out to the garage
dump it in the garbage can
walk back inside
wash your hands
plop back down on the couch
resume playing your video game (or whatever you were doing)
Even if you don't think in English, you still probably still thought of a similar set of actions, except in your favorite language. Regardless of the language you chose, or the exact steps you took, taking out the garbage is a series of actions that terminates in the garbage being outside, and you being back inside, because of the actions you took.
Our thoughts are filled with brave, fierce, passionate actions: we live, we breathe, we walk, we talk, we laugh, we cry, we hope, we fear, we eat, we drink, we stop, we go, we take out the garbage. Above all else, we are free to do and to act. If we were all just rocks sitting in the sun, life might still be OK, but we wouldn't be free. Our freedom comes precisely from our ability to do things.
Of course our thoughts are also filled with nouns. We eat nouns, and buy nouns from the store, and we sit on nouns, and sleep on them. Nouns can fall on your head, creating a big noun on your noun. Nouns are things, and where would we be without things? But they're just things, that's all: the means to an end, or the ends themselves, or precious possessions, or names for the objects we observe around around us. There's a building. Here's a rock. Any child can point out the nouns. It's the changes happening to those nouns that make them interesting.
Change requires action. Action is what gives life its spice. Action even gives spices their spice! After all, they're not spicy until you eat them. Nouns may be everywhere, but life's constant change, and constant interest, is all in the verbs.
And of course in addition to verbs and nouns, we also have our adjectives, our prepositions, our pronouns, our articles, the inevitable conjunctions, the yummy expletives, and all the other lovely parts of speech that let us think and say interesting things. I think we can all agree that the parts of speech each play a role, and all of them are important. It would be a shame to lose any of them.
Wouldn't it be strange if we suddenly decided that we could no longer use verbs?
Let me tell you a story about a place that did exactly that...
The Kingdom of Nouns
In the Kingdom of Javaland, where King Java rules with a silicon fist, people aren't allowed to think the way you and I do. In Javaland, you see, nouns are very important, by order of the King himself. Nouns are the most important citizens in the Kingdom. They parade around looking distinguished in their showy finery, which is provided by the Adjectives, who are quite relieved at their lot in life. The Adjectives are nowhere near as high-class as the Nouns, but they consider themselves quite lucky that they weren't born Verbs.
Because the Verb citizens in this Kingdom have it very, very bad.
In Javaland, by King Java's royal decree, Verbs are owned by Nouns. But they're not mere pets; no, Verbs in Javaland perform all the chores and manual labor in the entire kingdom. They are, in effect, the kingdom's slaves, or at very least the serfs and indentured servants. The residents of Javaland are quite content with this situation, and are indeed scarcely aware that things could be any different.
Verbs in Javaland are responsible for all the work, but as they are held in contempt by all, no Verb is ever permitted to wander about freely. If a Verb is to be seen in public at all, it must be escorted at all times by a Noun.
Of course "escort", being a Verb itself, is hardly allowed to run around naked; one must procure a VerbEscorter to facilitate the escorting. But what about "procure" and "facilitate?" As it happens, Facilitators and Procurers are both rather important Nouns whose job is is the chaperonement of the lowly Verbs "facilitate" and "procure", via Facilitation and Procurement, respectively.
The King, consulting with the Sun God on the matter, has at times threatened to banish entirely all Verbs from the Kingdom of Java. If this should ever to come to pass, the inhabitants would surely need at least one Verb to do all the chores, and the King, who possesses a rather cruel sense of humor, has indicated that his choice would be most assuredly be "execute".
The Verb "execute", and its synonymous cousins "run", "start", "go", "justDoIt", "makeItSo", and the like, can perform the work of any other Verb by replacing it with an appropriate Executioner and a call to execute(). Need to wait? Waiter.execute(). Brush your teeth? ToothBrusher(myTeeth).go(). Take out the garbage? TrashDisposalPlanExecutor.doIt(). No Verb is safe; all can be replaced by a Noun on the run.
In the more patriotic corners of Javaland, the Nouns have entirely ousted the Verbs. It may appear to casual inspection that there are still Verbs here and there, tilling the fields and emptying the chamber pots. But if one looks more closely, the secret is soon revealed: Nouns can rename their execute() Verb after themselves without changing its character in the slightest. When you observe the FieldTiller till(), the ChamberPotEmptier empty(), or the RegistrationManager register(), what you're really seeing is one of the evil King's army of executioners, masked in the clothes of its owner Noun.
Verbs in Neighboring Kingdoms
In the neighboring programming-language kingdoms, taking out the trash is a straightforward affair, very similar to the way we described it in English up above. As is the case in Java, data objects are nouns, and functions are verbs.2 But unlike in Javaland, citizens of other kingdoms may mix and match nouns and verbs however they please, in whatever way makes sense for conducting their business.
For instance, in the neighboring realms of C-land, JavaScript-land, Perl-land and Ruby-land, someone might model taking out the garbage as a series of actions — that is to say, verbs, or functions. Then if they apply the actions to the appropriate objects, in the appropriate order (get the trash, carry it outside, dump it in the can, etc.), the garbage-disposal task will complete successfully, with no superfluous escorts or chaperones required for any of the steps.
There's rarely any need in these kingdoms to create wrapper nouns to swaddle the verbs. They don't have GarbageDisposalStrategy nouns, nor GarbageDisposalDestinationLocator nouns for finding your way to the garage, nor PostGarbageActionCallback nouns for putting you back on your couch. They just write the verbs to operate on the nouns lying around, and then have a master verb, take_out_garbage(), that springs the subtasks to action in just the right order.
These neighboring kingdoms generally provide mechanisms for creating important nouns, when the need arises. If the diligent inventors in these kingdoms create an entirely new, useful concept that didn't exist before, such as a house, or a cart, or a machine for tilling fields faster than a person can, then they can give the concept a Class, which provides it with a name, a description, some state, and operating instructions.
The difference is that when Verbs are allowed to exist independently, you don't need to invent new Noun concepts to hold them.
Javalanders look upon their neighbors with disdain; this is the way of things in the Kingdoms of Programming.
If You Dig a Hole Deep Enough...
On the other side of the world is a sparsely inhabited region in whose kingdoms Verbs are the citizens of eminence. These are the Functional Kingdoms, including Haskellia, Ocamlica, Schemeria, and several others. Their citizens rarely cross paths with the kingdoms near Javaland. Because there are few other kingdoms nearby, the Functional Kingdoms must look with disdain upon each other, and make mutual war when they have nothing better to do.
In the Functional Kingdoms, Nouns and Verbs are generally considered equal-caste citizens. However, the Nouns, being, well, nouns, mostly sit around doing nothing at all. They don't see much point in running or executing anything, because the Verbs are quite active and see to all that for them. There are no strange laws mandating the creation of helper Nouns to escort each Verb, so there are only exactly as many Nouns as there are Things in each kindgom.
As a result of all this, the Verbs have the run of the place, if you'll pardon the expression. As an outsider, you could easily form the impression that Verbs (i.e., the functions) are the most important citizens by far. That, incidentally, is why they're called the Functional Kingdoms and not the Thingy Kingdoms.
In the remotest regions, beyond the Functional Kingdoms, lies a fabled realm called Lambda the Ultimate. In this place it is said that there are no nouns at all, only verbs! There are "things" there, but all things are created from verbs, even the very integers for counting lambs, which are the most popular form of trading currency there, if the rumors speak truth. The number zero is simply lambda(), and 1 is lambda(lambda()), 2 is lambda(lambda(lambda())), and so on. Every single Thing in this legendary region, be it noun, verb or otherwise, is constructed from the primal verb "lambda".3
To be quite honest, most Javalanders are blissfully unaware of the existence of the other side of the world. Can you imagine their culture shock? They would find it so disorienting that they might have to invent some new nouns (such as "Xenophobia") to express their new feelings.
Are Javalanders Happy?
You might think daily life in Javaland would be at best a little strange, and at worst grossly inefficient. But you can tell how happy a society is through their nursery rhymes, and Javaland's are whimsically poetic. For instance, Javaland children oft recite the famous cautionary tale:
For the lack of a nail,
throw new HorseshoeNailNotFoundException("no nails!");
For the lack of a horseshoe,
EquestrianDoctor.getLocalInstance().getHorseDispatcher().shoot();
For the lack of a horse,
RidersGuild.getRiderNotificationSubscriberList().getBroadcaster().run(
new BroadcastMessage(StableFactory.getNullHorseInstance()));
For the lack of a rider,
MessageDeliverySubsystem.getLogger().logDeliveryFailure(
MessageFactory.getAbstractMessageInstance(
new MessageMedium(MessageType.VERBAL),
new MessageTransport(MessageTransportType.MOUNTED_RIDER),
new MessageSessionDestination(BattleManager.getRoutingInfo(
BattleLocation.NEAREST))),
MessageFailureReasonCode.UNKNOWN_RIDER_FAILURE);
For the lack of a message,
((BattleNotificationSender)
BattleResourceMediator.getMediatorInstance().getResource(
BattleParticipant.PROXY_PARTICIPANT,
BattleResource.BATTLE_NOTIFICATION_SENDER)).sendNotification(
((BattleNotificationBuilder)
(BattleResourceMediator.getMediatorInstance().getResource(
BattleOrganizer.getBattleParticipant(Battle.Participant.GOOD_GUYS),
BattleResource.BATTLE_NOTIFICATION_BUILDER))).buildNotification(
BattleOrganizer.getBattleState(BattleResult.BATTLE_LOST),
BattleManager.getChainOfCommand().getCommandChainNotifier()));
For the lack of a battle,
try {
synchronized(BattleInformationRouterLock.getLockInstance()) {
BattleInformationRouterLock.getLockInstance().wait();
}
} catch (InterruptedException ix) {
if (BattleSessionManager.getBattleStatus(
BattleResource.getLocalizedBattleResource(Locale.getDefault()),
BattleContext.createContext(
Kingdom.getMasterBattleCoordinatorInstance(
new TweedleBeetlePuddlePaddleBattle()).populate(
RegionManager.getArmpitProvince(Armpit.LEFTMOST)))) ==
BattleStatus.LOST) {
if (LOGGER.isLoggable(Level.TOTALLY_SCREWED)) {
LOGGER.logScrewage(BattleLogger.createBattleLogMessage(
BattleStatusFormatter.format(BattleStatus.LOST_WAR,
Locale.getDefault())));
}
}
}
For the lack of a war,
new ServiceExecutionJoinPoint(
DistributedQueryAnalyzer.forwardQueryResult(
NotificationSchemaManager.getAbstractSchemaMapper(
new PublishSubscribeNotificationSchema()).getSchemaProxy().
executePublishSubscribeQueryPlan(
NotificationSchema.ALERT,
new NotificationSchemaPriority(SchemaPriority.MAX_PRIORITY),
new PublisherMessage(MessageFactory.getAbstractMessage(
MessageType.WRITTEN,
new MessageTransport(MessageTransportType.WOUNDED_SURVIVOR),
new MessageSessionDestination(
DestinationManager.getNullDestinationForQueryPlan()))),
DistributedWarMachine.getPartyRoleManager().getRegisteredParties(
PartyRoleManager.PARTY_KING ||
PartyRoleManager.PARTY_GENERAL ||
PartyRoleManager.PARTY_AMBASSADOR)).getQueryResult(),
PriorityMessageDispatcher.getPriorityDispatchInstance())).
waitForService();
All for the lack of a horseshoe nail.
It remains wonderful advice, even to this very day.
Although the telling of the tale in Javaland differs in some ways from Ben Franklin's original, Javalanders feel their rendition has a distinct charm all its own.
The main charm is that the architecture is there for all to see. Architecture is held in exceptionally high esteem by King Java, because architecture consists entirely of nouns. As we know, nouns are things, and things are prized beyond all actions in the Kingdom of Java. Architecture is made of things you can see and touch, things that tower over you imposingly, things that emit a satisfying clunk when you whack them with a stick. King Java dearly loves clunking noises; he draws immense satisfaction from kicking the wheels when he's trying out a new horse-drawn coach. Whatever its flaws may be, the tale above does not want for things.
One of our first instincts as human beings is to find shelter from the elements; the stronger the shelter, the safer we feel. In Javaland, there are many strong things to make the citizens feel safe. They marvel at the massive architectural creations and think "this must be a strong design". This feeling is reinforced when they try to make any changes to the structure; the architectural strength then becomes daunting enough that they feel nobody could bring this structure down.
In addition to the benefits of a strong architecture, everything in Javaland is nicely organized: you'll find every noun in its proper place. And the stories all take a definite shape: object construction is the dominant type of expression, with a manager for each abstraction and a run() method for each manager. With a little experience at this kind of conceptual modeling, Java citizens realize they can express any story in this style. There's a kind of "noun calculus" backing it that permits the expression of any abstraction, any computation you like. All one needs are sufficient nouns, constructors for those nouns, accessor methods for traversing the noun-graph, and the all-important execute() to carry out one's plans.
The residents of the Kingdom of Java aren't merely happy — they're bursting with pride!
StateManager.getConsiderationSetter("Noun Oriented Thinking", State.HARMFUL).run()
Or, as it is said outside the Kingdom of Java, "Noun Oriented Thinking Considered Harmful".
Object Oriented Programming puts the Nouns first and foremost. Why would you go to such lengths to put one part of speech on a pedestal? Why should one kind of concept take precedence over another? It's not as if OOP has suddenly made verbs less important in the way we actually think. It's a strangely skewed perspective. As my friend Jacob Gabrielson once put it, advocating Object-Oriented Programming is like advocating Pants-Oriented Clothing.
Java's static type system, like any other, has its share of problems. But the extreme emphasis on noun-oriented thought processes (and consequently, modeling processes) is more than a bit disturbing. Any type system will require you to re-shape your thoughts somewhat to fit the system, but eliminating standalone verbs seems a step beyond all rationale or reason.
C++ doesn't exhibit the problem, because C++, being a superset of C, allows you to define standalone functions. Moreover, C++ provides a distinct namespace abstraction; Java overloads the idea of a Class to represent namespaces, user-defined types, syntactic delegation mechanisms, some visibility and scoping mechanisms, and more besides.
Don't get me wrong; I'm not claiming C++ is "good". But I do find myself appreciating the flexibility of its type system, at least compared with Java's. C++ suffers from problems causing reasonable-looking sentences to cause listeners to snap and try to kill you (i.e., unexpected segfaults and other pitfalls for the unwary), and it can be extremely difficult to find the exact incantation for expressing a particular thought in C++. But the range of succinctly expressible thoughts far exceeds Java's, because C++ gives you verbs, and who'd want to speak in a language that doesn't?
Classes are really the only modeling tool Java provides you. So whenever a new idea occurs to you, you have to sculpt it or wrap it or smash at it until it becomes a thing, even if it began life as an action, a process, or any other non-"thing" concept.
I've really come around to what Perl folks were telling me 8 or 9 years ago: "Dude, not everything is an object."
It's odd, though, that Java4 appears to be the only mainstream object-oriented language that exhibits radically noun-centric behavior. You'll almost never find an AbstractProxyMediator, a NotificationStrategyFactory, or any of their ilk in Python or Ruby. Why do you find them everywhere in Java? It's a sure bet that the difference is in the verbs. Python, Ruby, JavaScript, Perl, and of course all Functional languages allow you to declare and pass around functions as distinct entities without wrapping them in a class.
It's certainly easier to do this in dynamically typed languages; you just pass a reference to the function, obtained from its name, and it's up to the caller to invoke the function with the proper arguments and use its return value correctly.
But many statically-typed languages have first-class functions as well. This includes verbosely-typed languages like C and C++, and also type-inferring languages like Haskell and ML. The languages just need to provide a syntax for creating, passing and invoking function literals with an appropriate type signature.
There's no reason Java couldn't simply add first-class functions and finally enter the grown-up, non-skewed world that allows people to use verbs as part of their thought processes. In fact there's a JVM language called The Nice programming language that sports a very Java-like syntax, but also includes expressive facilities for using verbs: standalone functions, which Java forces you to wrap with Callbacks or Runnables or other anonymous interface implementation classes to be able to refer to them.
Sun wouldn't even have to break their convention of requiring all functions to be "owned" by classes. Every anonymous function could carry an implicit "this" pointer to the class in which it was defined; problem solved.
I don't know why Sun insists on keeping Java squarely planted in the Kingdom of Nouns. I doubt it's a matter of underestimating their constituency; they added generics, which are a far more complex concept, so they clearly no longer care deeply about keeping the language simple. And that's not a bad thing, necessarily, because Java's established now: it makes more sense to start giving Java programmers tools that let them program the way they think.
I sure hope they fix this, so I can take the trash out and get back to my video game. Or whatever I was doing.
Notes
[1] Beginning with the verb "to stamp out", which is being replaced by a call to VerbEliminatorFactory.createVerbEliminator(currentContext).operate(). But that's getting waaaaay ahead of ourselves...
[2] And variable names are proper nouns, attributes are adjectives, operators often serve as conjunctions, varargs are the pronoun "y'all", and so on. But this is all beside the point of our story.
[3] The meaning of the verb "lambda" is allegedly "to lambda".
[4] And arguably C#, due to its similar roots.
Jumat, 24 Maret 2006
Moore's Law is Crap
Sometimes people ask me how I find time to go learn new stuff. Here's the answer: you make time.
Nobody ever likes that kind of answer, but we all know it's the real one.
My brother Dave put on some pretty serious weight after he graduated from high school. He'd gone from playing varsity football to working two jobs and going to college full time. It didn't help that one of his jobs was pizza delivery, and the other was waiting tables. Soon he was a fat, fat kid. Went from a buck-eighty to a deuce and a half, at least, maybe a deuce sixty.
One day he saw a truck with a bumper sticker on it that said: "Lose weight now, ask me how!" So he pulled up next to the truck at the next stoplight, and said to the two cowboy-types in it: "How do I lose weight?" They yelled back: "Just lose weight, ya fat pig! Haw haw haw HAW HAW HAW HAW!" and then drove off.
Dave was sad about this advice for a brief while, but eventually he brightened up, because he did know how to lose weight. Cripes, he'd been a champion varsity football player two years prior. It's not like there's any magic to it. He went and bought a mountain bike, started riding the crap out of it, joined a gym, watched what he ate, and lost about 85 pounds over the next year.
I was 145 pounds 2 years ago today, after a similar 2-year diet and exercise kick that took me down from a deuce and a half myself. Then I fell off the wagon — it happens — and put on 50 pounds over the next 2 years. It sucks, but there's no magic. Two months ago I finally started going to the gym every day, 7 days a week, and my legs are sore every day. My weight hasn't improved at all yet. But it will. You just have to work at these things consistently.
But you knew that, didn't you?
This isn't a self-help blog, by the way. I'm not in that business. I'm not here to help, since I don't actually know the answers. I'm just here to rant, and pose occasional questions. It's what I do, when I'm not doing other stuff.
I don't know why I blog. I'm just compelled; it just happens whether I like it or not. Don't read too much into my blogs. My opinions change from day to day. The only things I've learned, the only universal constants, are that I don't know very much, and that public whale explosions are just about the funniest thing human beings can experience during our stay on Earth. I don't know why that is, either.
Today's blog is truly a rant; I just need to get this particular one off my chest, so my gym partner Todd can listen to me rant about something different next week.
The Big Choice
We all have to choose how to spend the time we're given.
If you don't choose, it just slips right by you. I know. On a trip to Vegas not too long ago, I made a pit stop in a casino restroom, and as I was washing my hands, there was this older guy there, also washing his hands. On a whim, I asked, "Hey man, how old are you?"
His reply? "Seventy-two! I have a son: I remember the day he was born like it was yesterday! I was holding him just like so. Well, guess what, he turned 40 years old just last week! It goes by in a flash! Before long, you'll be lookin' at THIS!" He pointed at his wrinkled mug, and concluded his monologue with: "Haw, haw, haw! HAW HAW HAW *cough* *cough* HAW *cough* *hack* HAW HAW HAW HAW HAW!" and walked out. I think I made his day, although I can't exactly say he made mine.
When you graduate from college (or high school, for that matter), you have a simple choice facing you. You can either keep learning, or you can stop.
There is an almost unbelievably easy heuristic for knowing whether you're learning. It goes like this: no pain, no gain. Learning is hard. If it's easy, then you're coasting; you're not making yourself better at something fundamentally new that you couldn't do before.
In other words, it's just like working out. You've gotta mix it up. If you're sore every day, then you're getting good workouts (assuming you can tell the difference between "good" soreness and "injury" soreness; if you're not sure, go ask a professional.)
When you do study, there's a lot of appeal to studying what you already know, because it's less painful. And of course to become an expert at any field, you have to focus on it pretty hard for a long time. But cross-training is well established in sports; you don't typically become a world-class baseball player by just playing baseball all the time. You have to do other kinds of workouts and exercises to maximize your strength, agility and endurance gains.
Cross-training improves you every bit as rapidly in other disciplines. That includes programming. If you're cranking out code as easily as breathing, then if you're getting better at all, it's so gradual that you'd never notice it happening. You won't have great insights until you get new perspectives from working hard, even if only occasionally, at stuff other than what you already know.
Being in school full-time is an amazing luxury, one that's hard to appreciate when you're actually there, because learning is painful. But trust me on this one: it's even more painful when all you have is scraps of time here and there.
While you're in school, assuming you make a reasonable effort at applying yourself once in a while, you learn a fantastic amount, and you learn it at a fantastic rate. Later you'll learn at a slower rate; it's pretty much guaranteed. Non-educational activities will inevitably intrude and consume the majority of your time.
Hence the choice. After you graduate, you can either learn a little, or not at all.
If you're in the "not at all" camp, well you've made your choice, and I respect it. You'll probably be happier than I am. I'm tormented by how slowly I have to move as a programmer. I now believe programming languages have failed. All of them. The programming world is a giant body shop, and we're building huge pyramids with millions of years of hard labor. I believe that stuff should be easier than it is, and it pisses me off that most people are so content with the state of the art, because it means they're not helping make it better.
To me, mainstream languages have failed because they're still fundamentally serial, fundamentally early von Neumann, fundamentally single-box languages. They're all vying for the position of "least crappy", and the current winner (whether it's Python, Ruby, Lisp, name your favorite) is just that: the least crappy. Because they're all focused on finding more elegant ways to express mostly-serial computations for crap computers. That, or faking parallelism poorly with threads, if the language supports them at all.
Sure, there have been some interesting attempts at parallel languages. Erlang is one of the better-known ones, and it's actually quite cool. But Erlang has failed too, because you haven't heard of it.
Programming's Biggest Problem
Our industry is in a seriously ugly position, right now, as we speak. Most of the hardware designers are focused on keeping Moore's Law going, because that's where the money is. They want that doubling every 18 months. Today it's probably quite within our reach to get 10x every 18 months, if we'd agree to focus on parallelism (in the massively distributed computing sense.)
But programmers like XML, to the point of focusing on it to an unhealthy extent. Same with C++. And Java. They like these things because they work, and because they like to minimize the amount of crap they have to learn. Because learning is painful. Remember? You might think I've gone way off track, off the deep end, even, but this is the same thread.
Let's face it: a parallel language will have to be radically different if it's to break free of the von Neumann Turing-machine rat race we're in. If we move to cellular automata, or in fact any other parallel computational model that's resilient to node failures, then we'll need a new language, because the current serial languages will perform badly, or be horribly hard to manage, or both.
Cell or grid (or whatever) parallel computing will have a radically different internal economy. It'll need new data structures, new algorithms, new instruction sets, new everything. You do realize that John von Neumann was an economist, right? Among (many) other things, he was an economist, and it influenced the design of his first computer, that one right there on your desk.
The computing machine JvN built was created in an environment very similar to the one in the movie Apollo 13, where the folks at Houston had to build a carbon-dioxide remover out of exactly the free junk available on the spacecraft, and then explain it to the crew so they could build their own copy of it.
Johnny went out collected a bunch of engineers: materials engineers, electrical engineers, mechanical engineers, everyone who had some spare junk. They came up with a design for a computation machine that just barely sufficed, one that could be built out of the crap they had available at the time: vacuum tubes, magnetic drums, wire, duct tape.
As he was creating this thing, Johnny was focusing on what he was calling the "internal economy" of the resulting machine. Secondary storage accesses were painfully slow. Memory accesses were faster. Register accesses were very fast. And so on. So he designed representations, data structures, and algorithms that were appropriate for that particular machine, the one he was building from spare parts.
If he'd made the machine out of Brazilian rainforest army ants, or mechanical gears, or falling dominoes with marionettes to pick them up again, his data structures and algorithms would have been very, very different. There are some commonalities, sure — you need numbers, and arithmetic, and functions, and data storage, and sorting, and so on. But what's most efficient for army ant computers isn't going to be most efficient for vacuum tube computers.
You do realize you can make a computing machine out of just about anything, right? And they don't all have to work like Turing machines do. Turing was one of the greatest geniuses of the century, but he'd have been the first person to tell you that there are infinitely many machine designs capable of the same computations. His was just one model, and his professor's (which led to Lisp) was just one other model. But who's to say they're the best models?
Some computing machines are more efficient at certain computations than others. Some are more practical to build than others. Some are faster than others. Some are more robust, or more inherently parallel.
You do realize that your brain is such a machine, right? And that it's 100,000 times faster than today's computers at pattern-matching tasks, because while JvN's machine operates serially, your neurons can all fire independently.
Let me give you a hint: your brain's operating system isn't written in C++.
Is our industry ever going to get out of this amazing backwater of a gridlock, this evolutionary dead-end we're in, where we programmers act like army ants, churning out loops and SOAP calls and UML diagrams as if we're weaving cloth with the very fabric of the computational universe?
If it ever happens, and by God I hope to witness it in my lifetime, then the computers and languages and data structures and algorithms are all going to have to change simultaneously. Because a language optimized for a serial computer will perform like crap along some important dimension of performance, probably more than one. But we can't switch wholesale to parallel languages either, because they'll perform like crap on today's computers: again, for some value of "perform" that's not worth discussing here, but it'll be some form of either computer-performance or people-performance.
And programmers are nothing if not fanatically obsessed with performance. Kinda ironic, huh?
Half the irony stems from knowing that there are far more productive languages out there than the ones most of us are using. But most of them perform poorly on our hardware, because these languages are targeting meta-virtual machines, typically "defined" (informally) by the capabilities of the language itself. And if you're not targeting exactly the hardware you're on, the impedance mismatch will slow the language down.
That's the problem with most JVM languages other than Java: they need hardware (think ants! anything can be hardware!) to support operations like long-jumps and tail-call optimization, but the JVM doesn't export those facilities as part of its abstract machine definition.
Same goes for Lisp. It can't get the performance break it deserves because the hardware available today isn't a Lisp machine. They've built them, and I can assure you that C++ would be the loser slug of a language on a Lisp machine. But, alas, performance isn't the only thing programmers care about. They also care about not having to learn anything new.
That's the other half of the irony. Programmers are obsessed with performance, and they'll go to almost any length to fiddle with their algorithms and data representations in order to eek every last cycle and byte from their programs. Any length, that is, except for learning a new language on new hardware. Even if it would get them a thousand-X performance improvement for the same productivity. Or a 1000x productivity improvement for the same performance.
Because they don't want to learn anything hard. No gain, no pain, problem solved.
And that's where we're at. Moore's Law is crap. If we ever want to be 10x as productive and computationally efficient, let alone 1000x, then our whole computing model will have to change at once. Nothing else will do. The incremental approaches have all failed. It's got to be a revolutionary change.
If everything changes all at once, that's going to pose a bit of a problem for the folks on the Zero Learning curve, wouldn't you say? Don't freak out and mail me about this, either, because I'm a pessimist now, at least about this particular topic, and I doubt we'll ever get out of our rut. We're ignoring the First Law of Holes.
You do realize that John von Neumann spent the last 10 years of his life singlehandedly developing a theory of computing based on cellular automata? The computer you're reading this blog rant on was his frigging prototype! He was going to throw it out and make a better one! And then he died of cancer, just like my brother Dave did, just like so many people with so much more to give and so much more life to live. And we're not making headway on cancer, either, because our computers and languages are such miserable crap.
You have no idea the pain I feel when I sit down to program. I'm walking on razor blades and broken glass. You have no idea the contempt I feel for C++, for J2EE, for your favorite XML parser, for the pathetic junk we're using to perform computations today. There are a few diamonds in the rough, a few glimmers of beauty here and there, but most of what I feel is simply indescribable nausea.
Are you beginning to see why I prefer to work with programmers who stay on the Upward Curve after they get out of school? Because even while we're grubbing around in the dirt — just a sorry bunch of illiterate, innumerate cavemen, here in the very heart of the Stone Age of programming — at least these upward-curve programmers give me some hope. Hope that if something better comes along, they'll give it a try, a serious try, the old college try. Or hope, even, that they'll build that "something better" with me.
Fat chance. But hope can keep ya going for a good long while.
Baby Steps
It's all still fun, though. Broken glass and razor blades aren't so bad, when I think about how much worse my lot in life could be, had I been born in a different time or place. I've got it pretty good, and so have you, in all probability.
At my current job1 they feed us and massage us like Kobe cows, and I'm surrounded by unbelievably brilliant people, all way smarter than me, and we're doing great stuff together. Make no mistake: my blog whining is all relative to a totally imaginary future, one which in all likelihood, should it ever come to pass, will be filled with even more whining about totally imagined new futures. It's just in our nature to whine. But really, I have no complaints.
I put a lot of stock in fun. And family. And trying to live my life in such a way that I won't have any major regrets when the game's over. So there's the first part of my schedule: having fun.
If you want to be on an upward curve, just make some time for it, and make it a habit. That's all there is to it. It doesn't matter if you're trying to get better at programming, or math, or fitness, or flying kites, or even humanity's Number One Fear, even worse than the fear of Death: public speaking. You just work your way up, a little at a time.
I can't promise you any satisfaction from the upward curve. You'll get better at a lot of things, and you'll have plenty of interesting insights. You may even get a better job, or build some software that makes you famous, or just have more fun at what you do. But you won't have much time for television. Something will have to give. We all have to choose how to play our time, and it's a zero-sum game.
If, like me, you're dissatisfied with the current state of affairs, well, believe you me, you can find a lot of consolation in a book on math, or machine learning, or compiler construction, or on just about anything that promises to help in some small way.
You do have to learn to put up with the pain of learning, though, and make it a habit to push yourself a little each day.
As far as the actuals go, well, you'll just have to find an approach that works for you personally. You might only be able to devote one quiet hour a week to studying, but like unit testing, or working out, or brushing your teeth, it's better than not doing it at all.
Just try to have fun, and hopefully the rest will fall into place.
Notes
[1] Permit me to assure you that I do not ever speak in any way for my current employer. Like, don't you even think it for one second. They're them, I'm me, and let's leave it at that. In fact, I don't even work there. My friend does. And neither me nor my friend speaks for them. OK? OK.
Nobody ever likes that kind of answer, but we all know it's the real one.
My brother Dave put on some pretty serious weight after he graduated from high school. He'd gone from playing varsity football to working two jobs and going to college full time. It didn't help that one of his jobs was pizza delivery, and the other was waiting tables. Soon he was a fat, fat kid. Went from a buck-eighty to a deuce and a half, at least, maybe a deuce sixty.
One day he saw a truck with a bumper sticker on it that said: "Lose weight now, ask me how!" So he pulled up next to the truck at the next stoplight, and said to the two cowboy-types in it: "How do I lose weight?" They yelled back: "Just lose weight, ya fat pig! Haw haw haw HAW HAW HAW HAW!" and then drove off.
Dave was sad about this advice for a brief while, but eventually he brightened up, because he did know how to lose weight. Cripes, he'd been a champion varsity football player two years prior. It's not like there's any magic to it. He went and bought a mountain bike, started riding the crap out of it, joined a gym, watched what he ate, and lost about 85 pounds over the next year.
I was 145 pounds 2 years ago today, after a similar 2-year diet and exercise kick that took me down from a deuce and a half myself. Then I fell off the wagon — it happens — and put on 50 pounds over the next 2 years. It sucks, but there's no magic. Two months ago I finally started going to the gym every day, 7 days a week, and my legs are sore every day. My weight hasn't improved at all yet. But it will. You just have to work at these things consistently.
But you knew that, didn't you?
This isn't a self-help blog, by the way. I'm not in that business. I'm not here to help, since I don't actually know the answers. I'm just here to rant, and pose occasional questions. It's what I do, when I'm not doing other stuff.
I don't know why I blog. I'm just compelled; it just happens whether I like it or not. Don't read too much into my blogs. My opinions change from day to day. The only things I've learned, the only universal constants, are that I don't know very much, and that public whale explosions are just about the funniest thing human beings can experience during our stay on Earth. I don't know why that is, either.
Today's blog is truly a rant; I just need to get this particular one off my chest, so my gym partner Todd can listen to me rant about something different next week.
The Big Choice
We all have to choose how to spend the time we're given.
If you don't choose, it just slips right by you. I know. On a trip to Vegas not too long ago, I made a pit stop in a casino restroom, and as I was washing my hands, there was this older guy there, also washing his hands. On a whim, I asked, "Hey man, how old are you?"
His reply? "Seventy-two! I have a son: I remember the day he was born like it was yesterday! I was holding him just like so. Well, guess what, he turned 40 years old just last week! It goes by in a flash! Before long, you'll be lookin' at THIS!" He pointed at his wrinkled mug, and concluded his monologue with: "Haw, haw, haw! HAW HAW HAW *cough* *cough* HAW *cough* *hack* HAW HAW HAW HAW HAW!" and walked out. I think I made his day, although I can't exactly say he made mine.
When you graduate from college (or high school, for that matter), you have a simple choice facing you. You can either keep learning, or you can stop.
There is an almost unbelievably easy heuristic for knowing whether you're learning. It goes like this: no pain, no gain. Learning is hard. If it's easy, then you're coasting; you're not making yourself better at something fundamentally new that you couldn't do before.
In other words, it's just like working out. You've gotta mix it up. If you're sore every day, then you're getting good workouts (assuming you can tell the difference between "good" soreness and "injury" soreness; if you're not sure, go ask a professional.)
When you do study, there's a lot of appeal to studying what you already know, because it's less painful. And of course to become an expert at any field, you have to focus on it pretty hard for a long time. But cross-training is well established in sports; you don't typically become a world-class baseball player by just playing baseball all the time. You have to do other kinds of workouts and exercises to maximize your strength, agility and endurance gains.
Cross-training improves you every bit as rapidly in other disciplines. That includes programming. If you're cranking out code as easily as breathing, then if you're getting better at all, it's so gradual that you'd never notice it happening. You won't have great insights until you get new perspectives from working hard, even if only occasionally, at stuff other than what you already know.
Being in school full-time is an amazing luxury, one that's hard to appreciate when you're actually there, because learning is painful. But trust me on this one: it's even more painful when all you have is scraps of time here and there.
While you're in school, assuming you make a reasonable effort at applying yourself once in a while, you learn a fantastic amount, and you learn it at a fantastic rate. Later you'll learn at a slower rate; it's pretty much guaranteed. Non-educational activities will inevitably intrude and consume the majority of your time.
Hence the choice. After you graduate, you can either learn a little, or not at all.
If you're in the "not at all" camp, well you've made your choice, and I respect it. You'll probably be happier than I am. I'm tormented by how slowly I have to move as a programmer. I now believe programming languages have failed. All of them. The programming world is a giant body shop, and we're building huge pyramids with millions of years of hard labor. I believe that stuff should be easier than it is, and it pisses me off that most people are so content with the state of the art, because it means they're not helping make it better.
To me, mainstream languages have failed because they're still fundamentally serial, fundamentally early von Neumann, fundamentally single-box languages. They're all vying for the position of "least crappy", and the current winner (whether it's Python, Ruby, Lisp, name your favorite) is just that: the least crappy. Because they're all focused on finding more elegant ways to express mostly-serial computations for crap computers. That, or faking parallelism poorly with threads, if the language supports them at all.
Sure, there have been some interesting attempts at parallel languages. Erlang is one of the better-known ones, and it's actually quite cool. But Erlang has failed too, because you haven't heard of it.
Programming's Biggest Problem
Our industry is in a seriously ugly position, right now, as we speak. Most of the hardware designers are focused on keeping Moore's Law going, because that's where the money is. They want that doubling every 18 months. Today it's probably quite within our reach to get 10x every 18 months, if we'd agree to focus on parallelism (in the massively distributed computing sense.)
But programmers like XML, to the point of focusing on it to an unhealthy extent. Same with C++. And Java. They like these things because they work, and because they like to minimize the amount of crap they have to learn. Because learning is painful. Remember? You might think I've gone way off track, off the deep end, even, but this is the same thread.
Let's face it: a parallel language will have to be radically different if it's to break free of the von Neumann Turing-machine rat race we're in. If we move to cellular automata, or in fact any other parallel computational model that's resilient to node failures, then we'll need a new language, because the current serial languages will perform badly, or be horribly hard to manage, or both.
Cell or grid (or whatever) parallel computing will have a radically different internal economy. It'll need new data structures, new algorithms, new instruction sets, new everything. You do realize that John von Neumann was an economist, right? Among (many) other things, he was an economist, and it influenced the design of his first computer, that one right there on your desk.
The computing machine JvN built was created in an environment very similar to the one in the movie Apollo 13, where the folks at Houston had to build a carbon-dioxide remover out of exactly the free junk available on the spacecraft, and then explain it to the crew so they could build their own copy of it.
Johnny went out collected a bunch of engineers: materials engineers, electrical engineers, mechanical engineers, everyone who had some spare junk. They came up with a design for a computation machine that just barely sufficed, one that could be built out of the crap they had available at the time: vacuum tubes, magnetic drums, wire, duct tape.
As he was creating this thing, Johnny was focusing on what he was calling the "internal economy" of the resulting machine. Secondary storage accesses were painfully slow. Memory accesses were faster. Register accesses were very fast. And so on. So he designed representations, data structures, and algorithms that were appropriate for that particular machine, the one he was building from spare parts.
If he'd made the machine out of Brazilian rainforest army ants, or mechanical gears, or falling dominoes with marionettes to pick them up again, his data structures and algorithms would have been very, very different. There are some commonalities, sure — you need numbers, and arithmetic, and functions, and data storage, and sorting, and so on. But what's most efficient for army ant computers isn't going to be most efficient for vacuum tube computers.
You do realize you can make a computing machine out of just about anything, right? And they don't all have to work like Turing machines do. Turing was one of the greatest geniuses of the century, but he'd have been the first person to tell you that there are infinitely many machine designs capable of the same computations. His was just one model, and his professor's (which led to Lisp) was just one other model. But who's to say they're the best models?
Some computing machines are more efficient at certain computations than others. Some are more practical to build than others. Some are faster than others. Some are more robust, or more inherently parallel.
You do realize that your brain is such a machine, right? And that it's 100,000 times faster than today's computers at pattern-matching tasks, because while JvN's machine operates serially, your neurons can all fire independently.
Let me give you a hint: your brain's operating system isn't written in C++.
Is our industry ever going to get out of this amazing backwater of a gridlock, this evolutionary dead-end we're in, where we programmers act like army ants, churning out loops and SOAP calls and UML diagrams as if we're weaving cloth with the very fabric of the computational universe?
If it ever happens, and by God I hope to witness it in my lifetime, then the computers and languages and data structures and algorithms are all going to have to change simultaneously. Because a language optimized for a serial computer will perform like crap along some important dimension of performance, probably more than one. But we can't switch wholesale to parallel languages either, because they'll perform like crap on today's computers: again, for some value of "perform" that's not worth discussing here, but it'll be some form of either computer-performance or people-performance.
And programmers are nothing if not fanatically obsessed with performance. Kinda ironic, huh?
Half the irony stems from knowing that there are far more productive languages out there than the ones most of us are using. But most of them perform poorly on our hardware, because these languages are targeting meta-virtual machines, typically "defined" (informally) by the capabilities of the language itself. And if you're not targeting exactly the hardware you're on, the impedance mismatch will slow the language down.
That's the problem with most JVM languages other than Java: they need hardware (think ants! anything can be hardware!) to support operations like long-jumps and tail-call optimization, but the JVM doesn't export those facilities as part of its abstract machine definition.
Same goes for Lisp. It can't get the performance break it deserves because the hardware available today isn't a Lisp machine. They've built them, and I can assure you that C++ would be the loser slug of a language on a Lisp machine. But, alas, performance isn't the only thing programmers care about. They also care about not having to learn anything new.
That's the other half of the irony. Programmers are obsessed with performance, and they'll go to almost any length to fiddle with their algorithms and data representations in order to eek every last cycle and byte from their programs. Any length, that is, except for learning a new language on new hardware. Even if it would get them a thousand-X performance improvement for the same productivity. Or a 1000x productivity improvement for the same performance.
Because they don't want to learn anything hard. No gain, no pain, problem solved.
And that's where we're at. Moore's Law is crap. If we ever want to be 10x as productive and computationally efficient, let alone 1000x, then our whole computing model will have to change at once. Nothing else will do. The incremental approaches have all failed. It's got to be a revolutionary change.
If everything changes all at once, that's going to pose a bit of a problem for the folks on the Zero Learning curve, wouldn't you say? Don't freak out and mail me about this, either, because I'm a pessimist now, at least about this particular topic, and I doubt we'll ever get out of our rut. We're ignoring the First Law of Holes.
You do realize that John von Neumann spent the last 10 years of his life singlehandedly developing a theory of computing based on cellular automata? The computer you're reading this blog rant on was his frigging prototype! He was going to throw it out and make a better one! And then he died of cancer, just like my brother Dave did, just like so many people with so much more to give and so much more life to live. And we're not making headway on cancer, either, because our computers and languages are such miserable crap.
You have no idea the pain I feel when I sit down to program. I'm walking on razor blades and broken glass. You have no idea the contempt I feel for C++, for J2EE, for your favorite XML parser, for the pathetic junk we're using to perform computations today. There are a few diamonds in the rough, a few glimmers of beauty here and there, but most of what I feel is simply indescribable nausea.
Are you beginning to see why I prefer to work with programmers who stay on the Upward Curve after they get out of school? Because even while we're grubbing around in the dirt — just a sorry bunch of illiterate, innumerate cavemen, here in the very heart of the Stone Age of programming — at least these upward-curve programmers give me some hope. Hope that if something better comes along, they'll give it a try, a serious try, the old college try. Or hope, even, that they'll build that "something better" with me.
Fat chance. But hope can keep ya going for a good long while.
Baby Steps
It's all still fun, though. Broken glass and razor blades aren't so bad, when I think about how much worse my lot in life could be, had I been born in a different time or place. I've got it pretty good, and so have you, in all probability.
At my current job1 they feed us and massage us like Kobe cows, and I'm surrounded by unbelievably brilliant people, all way smarter than me, and we're doing great stuff together. Make no mistake: my blog whining is all relative to a totally imaginary future, one which in all likelihood, should it ever come to pass, will be filled with even more whining about totally imagined new futures. It's just in our nature to whine. But really, I have no complaints.
I put a lot of stock in fun. And family. And trying to live my life in such a way that I won't have any major regrets when the game's over. So there's the first part of my schedule: having fun.
If you want to be on an upward curve, just make some time for it, and make it a habit. That's all there is to it. It doesn't matter if you're trying to get better at programming, or math, or fitness, or flying kites, or even humanity's Number One Fear, even worse than the fear of Death: public speaking. You just work your way up, a little at a time.
I can't promise you any satisfaction from the upward curve. You'll get better at a lot of things, and you'll have plenty of interesting insights. You may even get a better job, or build some software that makes you famous, or just have more fun at what you do. But you won't have much time for television. Something will have to give. We all have to choose how to play our time, and it's a zero-sum game.
If, like me, you're dissatisfied with the current state of affairs, well, believe you me, you can find a lot of consolation in a book on math, or machine learning, or compiler construction, or on just about anything that promises to help in some small way.
You do have to learn to put up with the pain of learning, though, and make it a habit to push yourself a little each day.
As far as the actuals go, well, you'll just have to find an approach that works for you personally. You might only be able to devote one quiet hour a week to studying, but like unit testing, or working out, or brushing your teeth, it's better than not doing it at all.
Just try to have fun, and hopefully the rest will fall into place.
Notes
[1] Permit me to assure you that I do not ever speak in any way for my current employer. Like, don't you even think it for one second. They're them, I'm me, and let's leave it at that. In fact, I don't even work there. My friend does. And neither me nor my friend speaks for them. OK? OK.
Sabtu, 18 Maret 2006
The Truth About Interviewing
Warning: the title of this blog entry is very slightly misleading. It really ought to be called "The Partial (At Best) and In Any Case Utterly Biased So-Called 'Truth' About Certain Restricted Kinds of Technical Interviewing By a Complete Bigoted Snobby Jerk Who Doesn't Know What He's Talking About Becuz PHP r00lZ And You Don't Need To Know Anything Else!!!!"
However, that was a wee bit too long, so I shortened it. But consider yourself warned. (And yes, I like the longer title better too!)
Anyhoo...
An anonymous coward commented on one of my blogs:
In regards to the interview chronicles regarding how someone was aweful or saved themselves during job interviews, I'd have to say you're a bit of snob aren't you? Granted the people you plan to hire will be paid substantial amounts of money to do good solid work, I think you're the kind of interviewer who tends to mold the people who work under him into little clones of him. "You must know this for me to consider you a 'true engineer' otherwise, I will boot you out the door." I think it's narrow minded and it's a wonder why more people don't go into the technical fields with the people who are currently populating it. Egoists and snobs even if it is deserved.
<satire>
Well said! Except for the last sentence, which is missing a verb, not to mention any semblance of intelligibility. But other than that, I think we should give this commenter a position as the Editor-In-Chief of the New York Times.
As it happens, I'm in the same boat as the commenter. Really.
See, I've always thought it would be cool to be an airline pilot. But I went to interview for the position, and those damn snobs wanted me to be a miniature clone of their totally biased idea of an "ideal airline pilot." They expected me to know all this crapola about meteorology and electrical engineering and other stuff that obviously has nothing to do with flying a plane. Plus I had to have all these flight hours logged, I mean you just wouldn't believe the requirements. This was obviously just an "old boys" network trying to keep honest people like me out. I'm surprised anyone gets into that field at all.
So then I thought, maybe I should go be a brain surgeon. But they wouldn't even let me apply! They snobbily told me that I had to know a whole bunch more about brains, and surgery, and stuff. I've had high school biology, during which I dissected a frog, plus I'm generally a careful guy. I'm smart, and my hands don't shake much, and I can figure stuff out when I need to. And I was really motivated to do brain surgery. But they didn't consider me "true" brain surgeon material. Of all the nerve! I think those surgical interviewers are just cold-blooded snobs. They don't appreciate my true talents. They just want pathetic little clones of the existing brain surgeons out there.
Those hospital and airline directors aren't the only snobs out there, either, I can tell you from personal experience. See, I can juggle five balls pretty well (here's a short video of me doing it), and yet every time I apply for a position in Cirque du Soleil, I don't even get a reply from them! Talk about snobbery. They don't even write back. What's wrong with my juggling? Nothing! They just want a pretty little clone of Viktor Kee. Granted, they're going to pay their jugglers a lot of money to do good solid work, but I think they're the being fairly narrow-minded by not considering me.
Then I wrote a bunch of letters to Hollywood movie studios, offering to be an animal trainer, since I have a Shih Tzu named Cino, and I can get him to sit sometimes. I mean, that took a lot of work! That little dog hardly listens to a word I say. So I feel pretty qualified to be an animal trainer. Sure, I've never worked directly with a bear or a tiger before, but it can't be all that different. You just shout SIT! and then throw them a biscuit when they finally do it, right? Like, a bear-biscuit or a tiger-biscuit instead of a dog biscuit, but other than that it's gonna be the same, I just know it. But those animal-trainer egoists just booted me right out the door.
So I pondered it, and I thought: you know, I'm not going to go into a field full of egotistical airline pilots and brain surgeons and circus performers and animal trainers. They're just trying to sculpt little perfectly-moulded clones of themselves. Screw them!
I made sure to give them an earful by commenting on all their blogs, anonymously. That'll show 'em! I feel much better for telling them what a bunch of brain animal flying juggling surgery training JERKS they all are. I also left out some verbs, just to throw 'em for a loop. Deserved ego they it even wankers.
So yeah, Mr. Commenter. You've got it pegged. Snob city!
</satire>
A Minor (But Popular) Misconception
Before we get started with the real blog entry, let's get this cleared up: managers aren't the only people who interview you. Most interview loops are conducted by engineers who will be working on the same team as you. Nobody works for me, but my company still makes me interview people.
So we don't want little clones working for us; we want them working with us. </really-end-satire-now>
The Harsh Reality
If you want a job at a company like Microsoft, Yahoo!, Apple, or Amazon.com, they're going to have high standards. It doesn't matter if you "know how to program". They're going to test you on algorithmic complexity analysis, advanced data structures, algorithm design, searching and sorting, internationalization techniques, network protocols, OS-level memory management, parsing and semantic analysis, recursion and mathematical induction, graph theory, combinatorics, programming language theory, machine architecture, discrete math and logic, graphics and window systems, fonts and typesetting, color spaces and representations, databases and query languages, filesystems and storage, embedded systems, device drivers, mobile and wireless protocols, and internet standards and technologies.
If you're lucky, that is.
If you're unlucky, they'll ask you to derive the outline of their Ph.D. thesis on fault-tolerant massively parallel machine-learning systems. Or to solve a grand-unification style computation problem involving telephone switches, grid networks, and third-degree differential equations. Or, God forbid, they'll ask you about the darkest corners of C++ syntax.
And you want to know why they'll ask you about that stuff? Because they're using it every day. They've tried hiring people who don't know this stuff. Believe me, they try all the time. They want to hire more programmers, and they're out there on the constant lookout for new meat. But when they lower their standards, they get burned. The 747 crashes, the patient dies, the juggler drops a bowling pin on someone's head, the tiger rips someone's throat out. In the software world: the service goes down for days, losing them millions; the project gets delivered late or even not at all, losing them contracts and customers; they lose the business battle to competitors who hired better engineers.
Putting together a pretty Ruby on Rails site is no small feat. Learning to program is no small feat. Many people try and fail to get even that far. But it's 3- to 5-ball juggling, and it just doesn't cut it for the Cirque du Soleils and private hospitals of the software industry. I'm sure you want to be a racecar driver, a hang-gliding instructor, a corporate lawyer, a movie sound editor, a rocket scientist. But you know you don't have the requisite training or experience. Why do you think knowing a little (or even a lot) about programming automatically qualifies you to get hired at Microsoft?
Software companies have excruciatingly high standards, just like any other profession. Those that don't get eaten up by those that do.
Interviewers can't probe on all the important material in an interview — at least, not the way interviews are conducted in our industry today. So they use sampling. They ask a few breadth questions, to get a feel for how well you know the space we're in, and they ask a few depth questions, to see how well you can think on your feet, apply what you know, and come up with creative solutions to problems similar to the ones they encounter daily.
Or so the theory goes. Interviewing is necessarily fairly artificial. I'm not a big fan of the interviewing approach everyone uses, but there's nothing I personally can do about it. So I do my best to evaluate the software equivalents of brain surgeons and rocket scientists by chatting with them for 45 minutes. I have no other choice, nor does any other interviewer out there.
Well, that's not entirely true; there is one thing I can do about it: I can talk about it. I'm as open about the process as anyone I know, and I've blogged about it from time to time. I assure you that my interviewing style is not substantively different from the styles of thousands of interviewers at hundreds of high-tech companies.
If anything, I'm nicer, because I count motivation (that is, the desire to improve oneself) higher than any other factor in an interview. It's how I've managed to get hired at most of my jobs: my enthusiasm showed through, and was sufficient to cover my inevitable mistakes. So I tend to go a lot easier on candidates who show me they're on an upward curve, as opposed to just coasting.
But not all interviewers are nice, and I know lots and lots of interviewers, at many companies, who've decided that they can fully evaluate you based on whether you can solve some particular convex optimization problem (or graph-search problem, or logic problem, or whatever their pet Elephant Question is), and they ask every candidate this question regardless of their background or experience. In fact, I'd estimate that some 10% of all technical interviewers ask the same questions, year after year, and they could care less about your experience.
You should consider yourself lucky that I talk about this process at all, because most companies treat their interviewing strategy as some sort of proprietary secret, and it's hard to get information about it, unless you want to go interview at a bunch of places.
Hint to companies: you're all doing it the same way. Quit being so frigging secretive about it.
The reason I write about it is that I'm very interested in the whole process:
However, talking about it openly gets people amazingly upset. Many people seem to suffer from selective hearing in this space.
For one thing, people want jobs, and so any hint that someone out there, somewhere, might be asking questions to which they don't know the answers sends them into a spittle-emitting rage. Oddly, they don't seem to mind that they can't get hired as a brain surgeon or an animal trainer. But they think knowing a little programming means they should be a shoo-in for any tech company in the world, and anyone who asks something they don't know is a big mean jerk! (I actually discovered this lovely sentiment after posting recently to the O'Reilly Ruby Blog, so our Anonymous Coward is far from alone in his opinion.)
A friendly note to the Spittle People: Not all software development is the same. Think about that for a sec. You may be really good at web development, but that almost certainly doesn't qualify you to work on an OS kernel. Or you may be really good at OS kernels, but that almost certainly doesn't qualify you to be good at web development.
Our industry has fragmented, just the way the medical industry has, and there are different job roles now. Unless you understand this, you're going to be upset whenever a technical interviewer talks about something you're unfamiliar with. If so, take a deep breath and relax, because they're not talking about your skill set. They're talking about jugglers or hang gliding or something that sounds like what you do, but it's different. You can always go find an interviewer who will probe you on what you care about.
Another group of people who gets upset is other interviewers, because it's very much a stylistic thing. There are no proven techniques in interviewing, and personally I think it's something of a crapshoot. But most interviewers with a reasonable amount of experience seem to think they "have it down", and they think that when I talk about my style, I'm advocating that they change their style. And then they emit plenty of spittle too.
Well, wake up, angry people! There are hundreds of companies conducting technical interviews out there. They're doing it for different job roles, including DBAs, sysadmins, web programmers, unix programmers, windows programmers, java programmers, mobile-device programmers, and many other roles besides. For any given job role, they're all doing it about the same. It's not just me. In fact, those interviewers are YOU, once you get the job.
So if you think you can't get hired into a job you want, be it a software engineer at Boeing or an airline pilot for American, don't tell me about it. Whining at me isn't going to change the realities of our industry. Just go study whatever it is they want you to know.
Finding a Better Job
With all that said, you might wonder whether you want to work at a big software company in the first place. You need to think carefully about your motivations.
If you just want to make money, you can almost certainly find a job that's less technically demanding (or that's technically demanding in different ways) than working at companies like Microsoft. To cite just one of many, many possibilities: you can pick up enough PHP and MySQL to design websites, and go find a desperate hospital or insurance company or school system in need of a web presence, and no doubt it'll pay the bills just as well as Microsoft does.
If money's all you're after, you certainly don't need a computer science degree. Our industry desperately needs more programmers to crank out application code in C#, Java, PHP, Perl, C++, and other popular languages. If you go learn one or two of those languages, you're probably already a shoo-in for many companies.
You should be aware that working at most large tech companies isn't very glamorous, at least not as much as it seems from the outside. For instance, I know a lot of people who love video games, so they assume that game programming must be the most fun kind of programming out there. Then they go get jobs at a huge, mainstream game publisher, only to find that it's actually a bunch of tedious, schedule-driven work, just like the work at all other software companies.
The reality is, delivering good software just keeps getting harder, as customer expectations go up. That's why companies have to be so selective when they're interviewing. Big companies have serious challenges that aren't encountered by smaller companies, including the combinatorial communication (and dependencies) explosion, the need to maintain large legacy code bases, the problems of culture change as the company grows, and the need to find entirely new revenue streams in the face of pressure from Wall Street.
So most big companies are looking for a combination of skills: they need a mix of junior-ish people to help keep the business alive, and senior-ish people to try to help hoist them out of the holes they've dug for themselves internally by growing too fast.
Neither of those sounds very glamorous, does it? In practice, a startup is much more glamorous. You move faster; you're working on a leading-edge idea; you're burning as fast as you can to keep ahead of your competitors and your investor-imposed schedules. You can get brilliant things working, and worry about scaling them and maturing them later. All the fun part is in the startup days.
The trade-off is that you take on a great deal more risk. With a big company, you've got stability, and there's something to be said for that. A few failed startups can make just about anyone pretty risk-averse, at least for a while.
So you have to decide what you want. But like it or not, if you want to work as a software engineer for a big, top-tier, famous tech company, they're going to interview the living hell out of you. Even if you're really good, there's a chance you won't make it in the door.
You do have a certain amount of control over the situation, and this control is NOT something you exercise by commenting angrily on my blogs.
How do you participate in change? Well, first do some research. It's pretty easy to figure out what a company is interviewing for, and what the interviewers are going to be like. Just read the job descriptions on their website. They're usually pretty up-front, saying things like "B.S. in computer science, or equivalent", or they'll tell you exactly which tools and technologies they expect you to have mastered. If you're considering a particular company, then before you interview you should ask your recruiting contact there what kinds of questions to expect from the interview process.
If a company's going to ask you questions that you don't think matter, then don't work there! If you're right, then they'll go out of business, eventually. Or at least you can go throw your weight behind a company that you think has priorities aligned with your own. Vote with your feet!
Once you've actually started working somewhere, you'll eventually have to do interviews of your own. If you don't like their interviewing process, make a stink about it, and maybe they'll listen. Companies don't want false positives or false negatives any more than you do. If you have a great idea for improving the process, let them know. There are all kinds of minor variations you can try. Or maybe you can all agree to experiment with an entirely new process, and see how the resulting hires measure up.
I want to nitpick anyway!
Well, go ahead and comment; that's why I've opened this blog up for anonymous comments. So far, anyway. But blog comments don't actually carry much weight.
If you really want to make a difference, and start changing peoples' minds, go set up your own blog, and express your well-considered opinions there. It's free, and it's easy, and we ALL have access to self-publishing now. If you want to set up an alternate school of interviewing thought, well, I'm quite happy to read what you have to say. Seems like almost anything would be better than the mess we have with interviewing today.
The truth about interviewing is that everyone does it their own way, and it's always about the same: people interview for what they know. Nobody's very good at interviewing for skills they don't know, so the tendency to produce clones is actually quite widespread — the commenter had at least that much right. The best most of us can do is at least try to hire high-quality, generalist clones who pose the least risk of tiger throat-ripping software outages if they're hired.
But please: if you don't know something, don't go bersek on us. Just go read about it on Wikipedia. It's not as hard as you think!
However, that was a wee bit too long, so I shortened it. But consider yourself warned. (And yes, I like the longer title better too!)
Anyhoo...
An anonymous coward commented on one of my blogs:
In regards to the interview chronicles regarding how someone was aweful or saved themselves during job interviews, I'd have to say you're a bit of snob aren't you? Granted the people you plan to hire will be paid substantial amounts of money to do good solid work, I think you're the kind of interviewer who tends to mold the people who work under him into little clones of him. "You must know this for me to consider you a 'true engineer' otherwise, I will boot you out the door." I think it's narrow minded and it's a wonder why more people don't go into the technical fields with the people who are currently populating it. Egoists and snobs even if it is deserved.
<satire>
Well said! Except for the last sentence, which is missing a verb, not to mention any semblance of intelligibility. But other than that, I think we should give this commenter a position as the Editor-In-Chief of the New York Times.
As it happens, I'm in the same boat as the commenter. Really.
See, I've always thought it would be cool to be an airline pilot. But I went to interview for the position, and those damn snobs wanted me to be a miniature clone of their totally biased idea of an "ideal airline pilot." They expected me to know all this crapola about meteorology and electrical engineering and other stuff that obviously has nothing to do with flying a plane. Plus I had to have all these flight hours logged, I mean you just wouldn't believe the requirements. This was obviously just an "old boys" network trying to keep honest people like me out. I'm surprised anyone gets into that field at all.
So then I thought, maybe I should go be a brain surgeon. But they wouldn't even let me apply! They snobbily told me that I had to know a whole bunch more about brains, and surgery, and stuff. I've had high school biology, during which I dissected a frog, plus I'm generally a careful guy. I'm smart, and my hands don't shake much, and I can figure stuff out when I need to. And I was really motivated to do brain surgery. But they didn't consider me "true" brain surgeon material. Of all the nerve! I think those surgical interviewers are just cold-blooded snobs. They don't appreciate my true talents. They just want pathetic little clones of the existing brain surgeons out there.
Those hospital and airline directors aren't the only snobs out there, either, I can tell you from personal experience. See, I can juggle five balls pretty well (here's a short video of me doing it), and yet every time I apply for a position in Cirque du Soleil, I don't even get a reply from them! Talk about snobbery. They don't even write back. What's wrong with my juggling? Nothing! They just want a pretty little clone of Viktor Kee. Granted, they're going to pay their jugglers a lot of money to do good solid work, but I think they're the being fairly narrow-minded by not considering me.
Then I wrote a bunch of letters to Hollywood movie studios, offering to be an animal trainer, since I have a Shih Tzu named Cino, and I can get him to sit sometimes. I mean, that took a lot of work! That little dog hardly listens to a word I say. So I feel pretty qualified to be an animal trainer. Sure, I've never worked directly with a bear or a tiger before, but it can't be all that different. You just shout SIT! and then throw them a biscuit when they finally do it, right? Like, a bear-biscuit or a tiger-biscuit instead of a dog biscuit, but other than that it's gonna be the same, I just know it. But those animal-trainer egoists just booted me right out the door.
So I pondered it, and I thought: you know, I'm not going to go into a field full of egotistical airline pilots and brain surgeons and circus performers and animal trainers. They're just trying to sculpt little perfectly-moulded clones of themselves. Screw them!
I made sure to give them an earful by commenting on all their blogs, anonymously. That'll show 'em! I feel much better for telling them what a bunch of brain animal flying juggling surgery training JERKS they all are. I also left out some verbs, just to throw 'em for a loop. Deserved ego they it even wankers.
So yeah, Mr. Commenter. You've got it pegged. Snob city!
</satire>
A Minor (But Popular) Misconception
Before we get started with the real blog entry, let's get this cleared up: managers aren't the only people who interview you. Most interview loops are conducted by engineers who will be working on the same team as you. Nobody works for me, but my company still makes me interview people.
So we don't want little clones working for us; we want them working with us. </really-end-satire-now>
The Harsh Reality
If you want a job at a company like Microsoft, Yahoo!, Apple, or Amazon.com, they're going to have high standards. It doesn't matter if you "know how to program". They're going to test you on algorithmic complexity analysis, advanced data structures, algorithm design, searching and sorting, internationalization techniques, network protocols, OS-level memory management, parsing and semantic analysis, recursion and mathematical induction, graph theory, combinatorics, programming language theory, machine architecture, discrete math and logic, graphics and window systems, fonts and typesetting, color spaces and representations, databases and query languages, filesystems and storage, embedded systems, device drivers, mobile and wireless protocols, and internet standards and technologies.
If you're lucky, that is.
If you're unlucky, they'll ask you to derive the outline of their Ph.D. thesis on fault-tolerant massively parallel machine-learning systems. Or to solve a grand-unification style computation problem involving telephone switches, grid networks, and third-degree differential equations. Or, God forbid, they'll ask you about the darkest corners of C++ syntax.
And you want to know why they'll ask you about that stuff? Because they're using it every day. They've tried hiring people who don't know this stuff. Believe me, they try all the time. They want to hire more programmers, and they're out there on the constant lookout for new meat. But when they lower their standards, they get burned. The 747 crashes, the patient dies, the juggler drops a bowling pin on someone's head, the tiger rips someone's throat out. In the software world: the service goes down for days, losing them millions; the project gets delivered late or even not at all, losing them contracts and customers; they lose the business battle to competitors who hired better engineers.
Putting together a pretty Ruby on Rails site is no small feat. Learning to program is no small feat. Many people try and fail to get even that far. But it's 3- to 5-ball juggling, and it just doesn't cut it for the Cirque du Soleils and private hospitals of the software industry. I'm sure you want to be a racecar driver, a hang-gliding instructor, a corporate lawyer, a movie sound editor, a rocket scientist. But you know you don't have the requisite training or experience. Why do you think knowing a little (or even a lot) about programming automatically qualifies you to get hired at Microsoft?
Software companies have excruciatingly high standards, just like any other profession. Those that don't get eaten up by those that do.
Interviewers can't probe on all the important material in an interview — at least, not the way interviews are conducted in our industry today. So they use sampling. They ask a few breadth questions, to get a feel for how well you know the space we're in, and they ask a few depth questions, to see how well you can think on your feet, apply what you know, and come up with creative solutions to problems similar to the ones they encounter daily.
Or so the theory goes. Interviewing is necessarily fairly artificial. I'm not a big fan of the interviewing approach everyone uses, but there's nothing I personally can do about it. So I do my best to evaluate the software equivalents of brain surgeons and rocket scientists by chatting with them for 45 minutes. I have no other choice, nor does any other interviewer out there.
Well, that's not entirely true; there is one thing I can do about it: I can talk about it. I'm as open about the process as anyone I know, and I've blogged about it from time to time. I assure you that my interviewing style is not substantively different from the styles of thousands of interviewers at hundreds of high-tech companies.
If anything, I'm nicer, because I count motivation (that is, the desire to improve oneself) higher than any other factor in an interview. It's how I've managed to get hired at most of my jobs: my enthusiasm showed through, and was sufficient to cover my inevitable mistakes. So I tend to go a lot easier on candidates who show me they're on an upward curve, as opposed to just coasting.
But not all interviewers are nice, and I know lots and lots of interviewers, at many companies, who've decided that they can fully evaluate you based on whether you can solve some particular convex optimization problem (or graph-search problem, or logic problem, or whatever their pet Elephant Question is), and they ask every candidate this question regardless of their background or experience. In fact, I'd estimate that some 10% of all technical interviewers ask the same questions, year after year, and they could care less about your experience.
You should consider yourself lucky that I talk about this process at all, because most companies treat their interviewing strategy as some sort of proprietary secret, and it's hard to get information about it, unless you want to go interview at a bunch of places.
Hint to companies: you're all doing it the same way. Quit being so frigging secretive about it.
The reason I write about it is that I'm very interested in the whole process:
- I want to become a better interviewer myself.
- I want to improve the process overall, as it's far from perfect.
- I want to help candidates prepare better for their interviews.
However, talking about it openly gets people amazingly upset. Many people seem to suffer from selective hearing in this space.
For one thing, people want jobs, and so any hint that someone out there, somewhere, might be asking questions to which they don't know the answers sends them into a spittle-emitting rage. Oddly, they don't seem to mind that they can't get hired as a brain surgeon or an animal trainer. But they think knowing a little programming means they should be a shoo-in for any tech company in the world, and anyone who asks something they don't know is a big mean jerk! (I actually discovered this lovely sentiment after posting recently to the O'Reilly Ruby Blog, so our Anonymous Coward is far from alone in his opinion.)
A friendly note to the Spittle People: Not all software development is the same. Think about that for a sec. You may be really good at web development, but that almost certainly doesn't qualify you to work on an OS kernel. Or you may be really good at OS kernels, but that almost certainly doesn't qualify you to be good at web development.
Our industry has fragmented, just the way the medical industry has, and there are different job roles now. Unless you understand this, you're going to be upset whenever a technical interviewer talks about something you're unfamiliar with. If so, take a deep breath and relax, because they're not talking about your skill set. They're talking about jugglers or hang gliding or something that sounds like what you do, but it's different. You can always go find an interviewer who will probe you on what you care about.
Another group of people who gets upset is other interviewers, because it's very much a stylistic thing. There are no proven techniques in interviewing, and personally I think it's something of a crapshoot. But most interviewers with a reasonable amount of experience seem to think they "have it down", and they think that when I talk about my style, I'm advocating that they change their style. And then they emit plenty of spittle too.
Well, wake up, angry people! There are hundreds of companies conducting technical interviews out there. They're doing it for different job roles, including DBAs, sysadmins, web programmers, unix programmers, windows programmers, java programmers, mobile-device programmers, and many other roles besides. For any given job role, they're all doing it about the same. It's not just me. In fact, those interviewers are YOU, once you get the job.
So if you think you can't get hired into a job you want, be it a software engineer at Boeing or an airline pilot for American, don't tell me about it. Whining at me isn't going to change the realities of our industry. Just go study whatever it is they want you to know.
Finding a Better Job
With all that said, you might wonder whether you want to work at a big software company in the first place. You need to think carefully about your motivations.
If you just want to make money, you can almost certainly find a job that's less technically demanding (or that's technically demanding in different ways) than working at companies like Microsoft. To cite just one of many, many possibilities: you can pick up enough PHP and MySQL to design websites, and go find a desperate hospital or insurance company or school system in need of a web presence, and no doubt it'll pay the bills just as well as Microsoft does.
If money's all you're after, you certainly don't need a computer science degree. Our industry desperately needs more programmers to crank out application code in C#, Java, PHP, Perl, C++, and other popular languages. If you go learn one or two of those languages, you're probably already a shoo-in for many companies.
You should be aware that working at most large tech companies isn't very glamorous, at least not as much as it seems from the outside. For instance, I know a lot of people who love video games, so they assume that game programming must be the most fun kind of programming out there. Then they go get jobs at a huge, mainstream game publisher, only to find that it's actually a bunch of tedious, schedule-driven work, just like the work at all other software companies.
The reality is, delivering good software just keeps getting harder, as customer expectations go up. That's why companies have to be so selective when they're interviewing. Big companies have serious challenges that aren't encountered by smaller companies, including the combinatorial communication (and dependencies) explosion, the need to maintain large legacy code bases, the problems of culture change as the company grows, and the need to find entirely new revenue streams in the face of pressure from Wall Street.
So most big companies are looking for a combination of skills: they need a mix of junior-ish people to help keep the business alive, and senior-ish people to try to help hoist them out of the holes they've dug for themselves internally by growing too fast.
Neither of those sounds very glamorous, does it? In practice, a startup is much more glamorous. You move faster; you're working on a leading-edge idea; you're burning as fast as you can to keep ahead of your competitors and your investor-imposed schedules. You can get brilliant things working, and worry about scaling them and maturing them later. All the fun part is in the startup days.
The trade-off is that you take on a great deal more risk. With a big company, you've got stability, and there's something to be said for that. A few failed startups can make just about anyone pretty risk-averse, at least for a while.
So you have to decide what you want. But like it or not, if you want to work as a software engineer for a big, top-tier, famous tech company, they're going to interview the living hell out of you. Even if you're really good, there's a chance you won't make it in the door.
You do have a certain amount of control over the situation, and this control is NOT something you exercise by commenting angrily on my blogs.
How do you participate in change? Well, first do some research. It's pretty easy to figure out what a company is interviewing for, and what the interviewers are going to be like. Just read the job descriptions on their website. They're usually pretty up-front, saying things like "B.S. in computer science, or equivalent", or they'll tell you exactly which tools and technologies they expect you to have mastered. If you're considering a particular company, then before you interview you should ask your recruiting contact there what kinds of questions to expect from the interview process.
If a company's going to ask you questions that you don't think matter, then don't work there! If you're right, then they'll go out of business, eventually. Or at least you can go throw your weight behind a company that you think has priorities aligned with your own. Vote with your feet!
Once you've actually started working somewhere, you'll eventually have to do interviews of your own. If you don't like their interviewing process, make a stink about it, and maybe they'll listen. Companies don't want false positives or false negatives any more than you do. If you have a great idea for improving the process, let them know. There are all kinds of minor variations you can try. Or maybe you can all agree to experiment with an entirely new process, and see how the resulting hires measure up.
I want to nitpick anyway!
Well, go ahead and comment; that's why I've opened this blog up for anonymous comments. So far, anyway. But blog comments don't actually carry much weight.
If you really want to make a difference, and start changing peoples' minds, go set up your own blog, and express your well-considered opinions there. It's free, and it's easy, and we ALL have access to self-publishing now. If you want to set up an alternate school of interviewing thought, well, I'm quite happy to read what you have to say. Seems like almost anything would be better than the mess we have with interviewing today.
The truth about interviewing is that everyone does it their own way, and it's always about the same: people interview for what they know. Nobody's very good at interviewing for skills they don't know, so the tendency to produce clones is actually quite widespread — the commenter had at least that much right. The best most of us can do is at least try to hire high-quality, generalist clones who pose the least risk of tiger throat-ripping software outages if they're hired.
But please: if you don't know something, don't go bersek on us. Just go read about it on Wikipedia. It's not as hard as you think!
Langganan:
Postingan (Atom)