Esmerelda's Imagination

An actress acquaintance of mine—let's call her Esmerelda—once said, "I can't imagine being anything except an actress." To which the retort was given, "You can't be much of an actress then, can you?"

I was reminded of this exchange when someone said to me about Go, "I can't imagine programming in a language that doesn't have generics." My retort, unspoken this time, was, "You can't be much of a programmer, then, can you?"

This is not an essay about generics (which are a fine thing and may arrive in Go one day, or may not) but about imagination, or at least what passes for imagination among computer programmers: complaint. A friend observed that the definitive modern pastime is to complain on line. For the complainers, it's fun, for the recipients of the complaint it can be dispiriting. As a recipient, I am pushing back—by complaining, of course.

Not so long ago, a programmer was someone who programs, but that seems to be the last thing programmers do nowadays. Today, the definition of a programmer is someone who complains unless the problem being solved has already been solved and whose solution can be expressed in a single line of code. (From the point of view of a language designer, this reduces to a corollary of language success: every program must be reducible to single line of code or your language sucks. The lessons of APL have been lost.)

A different, more liberal definition might be that a programmer is someone who approaches every problem exactly the same way and complains about the tools if the approach is unsuccessful.

For the programmer population, the modern pastime demands that if one is required to program, or at least to think while programming, one blogs/tweets/rants instead. I have seen people write thousands of words of on-line vituperation that problem X requires a few extra keystrokes than it might otherwise, missing the irony that had they spent those words on programming, they could have solved the problem many times over with the saved keystrokes. But, of course, that would be programming.

Two years ago Go went public. This year, Dart was announced. Both came from Google but from different teams with different goals; they have little in common. Yet I was struck by a property of the criticisms of Dart in the first few days: by doing a global substitution of "Go" for "Dart", many of the early complaints about Go would have fit right into the stream of Dart invective. It was unnecessary to try Go or Dart before commenting publicly on them; in fact, it was important not to (for one thing, trying them would require programming). The criticisms were loud and vociferous but irrelevant because they weren't about the languages at all. They were just a standard reaction to something new, empty of meaning, the result of a modern programmer's need to complain about everything different. Complaints are infinitely recyclable. ("I can't imagine programming in a language without XXX.") After all, they have a low quality standard: they need not be checked by a compiler.

A while after Go launched, the criticisms changed tenor somewhat. Some people had actually tried it, but there were still many complainers, including the one quoted above. The problem now was that imagination had failed: Go is a language for writing Go programs, not Java programs or Haskell programs or any other language's programs. You need to think a different way to write good Go programs. But that takes time and effort, more than most will invest. So the usual story is to translate one program from another language into Go and see how it turns out. But translation misses idiom. A first attempt to write, for example, some Java construct in Go will likely fail, while a different Go-specific approach might succeed and illuminate. After 10 years of Java programming and 10 minutes of Go programming, any comparison of the language's capabilities is unlikely to generate insight, yet here come the results, because that's a modern programmer's job.

It's not all bad, of course. Two years on, Go has lots of people who've spent the time to learn how it's meant to be used, and for many willing to invest such time the results have been worthwhile. It takes time and imagination and programming to learn how to use any language well, but it can be time well spent. The growing Go community has generated lots of great software and has given me hope, hope that there may still be actual programmers out there.

However, I still see far too much ill-informed commentary about Go on the web, so for my own protection I will start 2012 with a resolution:

I resolve to recognize that a complaint reveals more about the complainer than the complained-about. Authority is won not by rants but by experience and insight, which require practice and imagination. And maybe some programming.

User experience

[We open in a well-lit corporate conference room. A meeting has been running for a while. Lots has been accomplished but time is running out.]

[The door opens and a tall, tow-headed twenty-something guy in glasses walks in, carrying a Mac Air and a folder.]

Manager:

Oh, here he is. This is Richard. I asked him to join us today. Glad he could make it. He's got some great user experience ideas.

Richard:

Call me Dick.

Manager:

Dick's done a lot of seminal UX work for us.

Engineer:

Hey, aren't you the guy who's arguing we shouldn't have search in e-books?

Dick:

Absolutely. It's a lousy idea.

Engineer:

What?

Dick:

Books are the best UI ever created. They've been perfected over more than 500 years of development. We shouldn't mess with success.

Product manager:

Well, this is a new age. We should be allowed to ...

Dick:

Books have never had search. If we add search, we'll just confuse the user.

Product manager:

Oh, you're right. We don't want to do that.

Engineer:

But e-books aren't physical books. They're not words on paper. They're just bits, information.

Dick:

Our users don't know that.

Engineer:

Yes they do! They don't want simple books, they want the possibilities that electronic books can bring. Do you know about information theory? Have you even heard of Claude Shannon?

Dick:

Isn't he the chef at that new biodynamic tofu restaurant in North Beach?

Engineer:

Uhh, yeah, OK. But look, you're treating books as a metaphor for your user interface. That's as lame as using a trash can to throw away files and folders. We can do so much more!

Dick:

You misunderstand. Our goal is to make computers easier to use, not to make them more useful.

Product manager:

Wow, that's good.

Engineer:

Wow.

Manager:

Let's get back on track. Dick, you had some suggestions for us?

Dick:

Yeah. I was thinking about the work we did with the Notes iPhone app. Using a font that looked like a felt marker was a big help for users.

Engineer:

Seriously?

Dick:

Yes, it made users feel more comfortable about keeping notes on their phone. Having a font that looks like handwriting helps them forget there's a computer underneath.

Engineer:

I see....

Dick:

Yes, so... I was thinking for the Address Book app for Lion, we should change the look to be like a...

Manager:

Can you show us?

Dick:

Yeah, sure. I have a mock-up here.

[Opens laptop, turns it to face the room.]

Product manager:

An address book! That's fantastic. Look at the detail! Leather, seams at the corners, a visible spine. This is awesome!

Engineer:

It's just a book. It's a throwback. What are you doing? Why does it need to look like a physical address book?

Dick:

Because it is an address book!

Engineer:

No it's not, it's an app!

Dick:

It's a book.

Engineer:

You've made it one. This time it's not even a metaphor - it's literally a book. You're giving up on the possibility of doing more.

Dick:

As I said, users don't care about functionality. They want comfort and familiarity. An Address Book app that looks like an address book will be welcome. Soothing.

Engineer:

If they want a paper address book, they can buy one.

Dick:

Why would they do that if they have one on their desktop?

Engineer:

Can they at least change the appearance? Is there a setting somewhere?

Dick:

Oh, no. We know better than the user - otherwise why are we here? Settings are just confusing.

Engineer:

I ... I really don't understand what's going on.

Manager:

That's OK, you don't have to, but I'd like to give you the action item to build it. End of the quarter OK?

Engineer:

Uhhh, sure.

Manager.

Dick, do you have the requirements doc there?

Dick:

Right here.

[Pushes the folder across the desk.]

Engineer:

Can't you just mail it to me?

Dick:

It's right there.

Engineer:

I know, but... OK.

Manager:

That's a great start, Dick. What else do you have?

Dick:

Well, actually, maybe this is the time to announce that I'm moving on. Today is my last day here.

Manager, Product manager:

[Unison] Oh no!

Dick:

Yeah, sorry about that. I've had an amazing time here changing the world but it's tiem for me to seek new challenges.

Manager:

Do you have something in mind?

Dick:

Yes, I'm moving north. Microsoft has asked me to head a group there. They've got some amazing new ideas around paper clips.

FADE

Regular expressions in lexing and parsing

Comments extracted from a code review. I've been asked to disseminate them more widely.

I should say something about regular expressions in lexing and
parsing. Regular expressions are hard to write, hard to write well,
and can be expensive relative to other technologies. (Even when they
are implemented correctly in N*M time, they have significant
overheads, especially if they must capture the output.)

Lexers, on the other hand, are fairly easy to write correctly (if not
as compactly), and very easy to test. Consider finding alphanumeric
identifiers. It's not too hard to write the regexp (something like
"[a-ZA-Z_][a-ZA-Z_0-9]*"), but really not much harder to write as a
simple loop. The performance of the loop, though, will be much higher
and will involve much less code under the covers. A regular expression
library is a big thing. Using one to parse identifiers is like using a
Ferrari to go to the store for milk.

And when we want to adjust our lexer to admit other character types,
such as Unicode identifiers, and handle normalization, and so on, the
hand-written loop can cope easily but the regexp approach will break
down.

A similar argument applies to parsing. Using regular expressions to
explore the parse state to find the way forward is expensive,
overkill, and error-prone. Standard lexing and parsing techniques are
so easy to write, so general, and so adaptable there's no reason to
use regular expressions. They also result in much faster, safer, and
compact implementations.

Another way to look at it is that lexers and parsing are matching
statically-defined patterns, but regular expressions' strength is that
they provide a way to express patterns dynamically. They're great in
text editors and search tools, but when you know at compile time what
all the things are you're looking for, regular expressions bring far
more generality and flexibility than you need.

Finally, on the point about writing well. Regular expressions are, in
my experience, widely misunderstood and abused. When I do code reviews
involving regular expressions, I fix up a far higher fraction of the
regular expressions in the code than I do regular statements. This is
a sign of misuse: most programmers (no finger pointing here, just
observing a generality) simply don't know what they are or how to use
them correctly.

Encouraging regular expressions as a panacea for all text processing
problems is not only lazy and poor engineering, it also reinforces
their use by people who shouldn't be using them at all.

So don't write lexers and parsers with regular expressions as the
starting point. Your code will be faster, cleaner, and much easier to
understand and to maintain.

Know your science

Except for the TV show "The Big Bang Theory", popular culture gets science wrong. We all know that.

But there's a way it tends to get science wrong that upsets me more than most. That is when it misuses the tools of science by willfully ignoring what science actually means.

One common example is celebrity equations, wherein some mathematical-looking expression mixes two or more celebrities together, as in (I'm making this one up and I'm not a cultural critic, let alone a comic, so please bear with me): Lady Gaga = (2*Madonna + Carrot Top)/3. Mathematically savvy readers will recognize that I normalized that equation. If you don't know what that means, you shouldn't be writing celebrity equations, because mathematical equations mean something, they're not just symbols. Like musical comedy based on bad notes, bogus mathematical equations are not funny, just lazy.

Some years ago I even wrote a letter to Entertainment Weekly when they had a long article full of egregious celebrity equations. To their credit, they published the letter and even mended their ways for a while. I quote the letter here:

According to EW math, the more buzz or intelligence you have, the less likely you are to be on the It List. That may be true, but I bet you didn't mean that. Your equation is art-directed nonsense. EW seems to think the joke is that the equations look cute: If Einstein is funny, his square root is hilarious...

In short, mathematics may look funny if you don't understand it but that doesn't make it funny if you misuse it in ignorance.

Another sort of abuse is comedy periodic tables: periodic tables of the vegetables, period table of the desserts, periodic table of the presidents, and on and on. There are zillions of them. I believe the vegetables one was the first widely distributed example.

What's wrong with them? Again, they miss the point about the one true periodic table, Mendeleev's periodic table of the elements. In fact, to put things with no structure into a periodic table not only misses the point of the periodic table, it misses the profound idea that some things have periods.

Mendeleev's table, by recognizing the periodic structure of the elements, predicted not only properties of the elements, but the very existence of undiscovered elements. It was a breakthrough.

The periodic table is not some artistic layout of letters, it's science at its very best, one of the great results of the 19th century and the birth of modern chemistry. It doesn't honor science to take, say, typefaces and put them in a funny-looking grid. That just mocks the idea that science can predict the way the world works.

Science is not arbitrary. Making arbitrary cultural artifacts by abusing scientific ideas is not just wrong, it's offensive. It cheapens science.

Another area of abuse is quantum mechanics, and a common victim is Heisenberg's uncertainty principle. Despite what some ill-informed academics would have you believe, Heisenberg's principle is not some general statment about weird shit happening in the world, it is a fantastically precise scientific statement about the limits of measurement of two simultaneous physical properties: position and momentum. It's not a metaphor!

What's really sad is that many of the commonest misuses of the terminology of quantum mechanics come from other areas of science and technology. For instance, there is a term in computer engineering called a Heisenbug, which refers to faults that are unpredictable, most often for bugs that go away when you examine them. It's a cute name but it isn't even a correct reference. The quantum mechanical property of things changing when you observe them is not the Heisenberg uncertainty principle, it's the observer effect. These two ideas are often confused but they are not the same. They're not even closely related.

The observer effect in quantum mechanics describes how the act of measuring a quantum system forces the system to cough up a measurable quantity, which triggers a "wave function collapse". Heisenberg's uncertainty principle says that the minimum product of the error in simultaneous measurement of a particle's position and momentum is Planck's constant divided by 4π, or as we write it in physics, ℏ/2. (By the way, that's an extremely small value.)

Not only are these very different ideas, neither of them has anything to do with computer bugs. The term Heisenbug is trendy but bogus and ignores some strange and beautiful ideas. It's no better informed than the square root of Einstein or the periodic table of the typefaces.

If you're going to use the terms of science to inform your world, please make a point to understand the science too. Your world will be richer for it.

MacDonald's not McDonald's

Same general concept. Had a hamburger here too.

When I travel I like to have a hamburger at a McDonald's restaurant. There are a number of these to be found around the world.

I can't find this on the web, so here follows a note I wrote in 1991
of an odd event in my computing career.

===

There's no such thing as bad publicity, the saying goes.
Unless, of course, you have no interest in being a public figure.
That's where Richard Stallman and I start to diverge.

A friend called early Saturday to report some good news; groggily,
I countered with a feeling of impending doom. I had to spend the
day preparing for a talk Monday afternoon at MIT. The talk itself
would be fine: I'd been invited by Butler Lampson to talk about Plan 9
at the Lab for Computer Science and took that as the kind of honor
I rise to. I'd give a hell of talk, provided I was allowed to.
The problem, I said, is that MIT and I are connected by some history.

I explained about The League for Programming Freedom,
just the `League' to insiders, whose multiply ambiguous name
hearkens to an innocent age in software. Free Software is
like Free Love, a hippie pipe dream in which
computing is free from venality, commercial interests, even
capitalism. The founder of the League, Richard Stallman,
has been preaching the gospel of promiscuous programming
for years now and has won many converts, especially
in academia. Especially at MIT. Especially among the undergraduates.
Especially in the building on Technology Square in Cambridge that
the League shares, in part, with LCS.

Stallman's sermons, in print and in person, always include a
long harangue about patents on software. The citation in that
harangue is usually #4,555,755, AT&T's US patent on what is
colloquially called ``backing store,'' a technique for implementing
windows on a bitmap display. This patent is of particular interest
to Stallman because he claims to have used the idea, before the patent
was filed, while writing the window system for the
Lisp Machine at MIT. The patent is of interest to me because I am
listed as the inventor.

I don't wish to dive into the legal and technical intricacies of
the patent question here. I will just state that I know what I
invented and when I invented it, and I know what Stallman did and
when he did it, and I do not believe that Stallman had nearly as
good an idea as I did. And as for the propriety of software
patents, I signed a contract when I joined AT&T that said, in
effect, that I could work on whatever I fancied and would be
supported well in that endeavor; all AT&T asked in return was that
they be allowed to make money from my work. In Stallman's
world that is a Faustian deal, but then I always thought only
boring people went to Heaven.

When the patent was filed (October 7, 1982; issued
November 26, 1985), there were very few software patents and
the occasion was celebrated. I was congratulated warmly and
people were excited about the future of software patents.
Nowadays, however, the climate in universities at least is
very different, and Richard Stallman is almost single-handedly
responsible for the change. (The business community, on the other
hand, is still excited.)

A few years ago, AT&T began quietly pressing its case on a small
portfolio of computer graphics patents, with #4,555,775 being central.
Polite letters were written to a number of places inviting people
to draw up license agreements; polite letters were returned, and
legalities proceeded normally. (I don't know, and wouldn't say if
I did, what the state of those legalities is today.) One of the
letters went to the X consortium at MIT, where it was largely
ignored. A follow-up letter early in 1991 hit the electronic
bulletin boards, however, and I have been a public figure ever
since.

I hung up and began preparing for the worst. Stallman and I had
never met, and I felt sure he'd capitalize on my visit to his
building.

I was therefore not surprised early Monday afternoon, as we were eating
a takeout lunch in the fifth-floor lounge at LCS, when someone
said that Stallman was preparing to stage a protest at my talk.
Jerry Saltzer went to his office to get a copy of the announcement,
made on a local LCS electronic bulletin board:

---

Date: Sun, 17 Nov 91 20:26:37 -0500
From: Richard Stallman
To: bboard@lcs.mit.edu
Subject: Protest the AT&T backing store patent, Monday afternoon

You may have heard that AT&T has a patent on a simple technique called
"backing store" which consists of saving the hidden parts of a window
in off-screen memory. AT&T is using this patent to threaten to sue
all the users of X windows, including MIT. A few weeks ago, the X
consortium stated that these threats are "chilling to university
research".

Rob Pike, who obtained this patent for AT&T, is going to be visiting
Tech Square on Monday afternoon. If you don't like AT&T's
monopolistic threat, now's the time to express your opinion by joining
in a quiet protest against his visit.

Let's meet on the fourth floor of 545 Tech Square, near room 430,
around 2:15pm.

Please make a sign, even an el-cheapo sign, to identify yourself as
part of the protest. Make up a slogan on the subject of Pike, AT&T,
backing store, X windows, patents..., then write it with a magic
marker on a piece of copier paper.

(Pike has spoken publicly in favor of software patents and the backing
store patent in particular. He is thus not a reluctant participant in
AT&T's campaign of threats.)

---

This notice has some characteristic inaccuracies, of which the worst
is that AT&T has never threatened to sue anyone over the patent,
but I love it anyway. The penultimate paragraph typifies Stallman
as only Stallman himself can. I asked if Stallman really believed
that people needed instructions on how to make signs. I was told
that among MIT undergraduates a bizarre form of political correctness
had developed, putting Stallman in charge of a pack of eager
misguided nerds who in a healthier environment would probably
be protesting the killing of rats in biology class.

My hosts at LCS were mortified. Stallman had staged a protest
at a company (to complain about Lotus's ``look and feel'' lawsuit),
but had never picketed a technical talk. Moreover, I was the
guest of LCS, not the League, had been invited to give this talk,
and was planning to give a technical lecture, not a legal debate,
on a topic unrelated to patents. There was even a suggestion
about examining the MIT code to see what it said about freedom
of speech. But then someone pointed out that
Stallman, for all his eccentricities (don't get me started) was
polite, and if he wanted a `quiet' protest it would be quiet, and
I agreed.
A wag remarked that in the cloistered MIT world, participating
in a demonstration like this would be a broadening experience.

I was admonished not to talk to the protesters, not to answer
any questions about patents, and to let one of my hosts deflect
any verbal missile hurled at me. I replied that I had expected
as much and was prepared. For example, I was wearing a Bugs Bunny
T-shirt rather than a three-piece suit; I am a researcher more
than an AT&T Ambassador, to quote a giveaway pencil from a few
years back.

I needed to go to the lecture room early to prepare a laptop and
overhead projector for a brief demo during my talk. Because of
a scheduling conflict, the room was not a classroom but a sort
of playroom full of bean bags and not enough chairs.

I quickly identified the protesters: they were the ones in the back
with hand-written slogans on pieces of copier paper taped to their
chests. ``We don't back AT&T's Store of Patents,'' was perhaps
the most creative sign, except for a woman in a wheelchair with a
large placard reading, ``Patents Cripple Software.'' I looked
twice to verify that she needed the wheelchair. I tried to meet
Stallman's gaze but he would only steal surreptitious sideways
glances as he nervously paced back and forth in front of his
entourage.

My equipment set up easily, so I sat down in a reserved chair
in the front row to await the starting time and began chatting
to someone from LCS. Someone tapped my shoulder. I turned to
see a sign-encrusted protester. Physical contact. I braced myself.
He spoke.

- Excuse me, would you mind moving? I won't be able to see the screen.

- Don't worry, I'm giving the talk so I'll be moving all through it.

- Fine. Thanks.

He sat down. At that moment, I finally relaxed with the realization
that nothing ugly was going to happen.

And nothing did. The talk went very well; I was pleased with the
story I told, a technical explanation of how distributed applications
are built in Plan 9 using its namespace operations (patent applied for).
The protesters were surprised, I think, that my subject was interesting
to them. At one point they all applauded spontaneously when I described
a feature of the system. I also think they were surprised that the
inventor of #4,555,755 was funny, theatrical, and clever. At least
half the questions (all technical) during and after the talk were from
the protesters. Stallman said nothing.

Afterwards an LCS member said that, as a result of Stallman's ploy,
my audience was about twice what it would have been. The people
he dragged in came for political reasons but ended up learning
something.

Here's what the League's newsletter said about the event:

---

CAMBRIDGE, MA, November 18, 1991 -- Rob Pike, a software designer
from AT&T Bell Labs, expected to deliver an ordinary seminar on
his latest research project. Instead, he found a room filled
with programmers carrying signs to protest the consequences of
his previous project: the AT&T "backing store" patent which AT&T
has used to threaten all the members of the X Consortium,
including MIT itself.
Of the approximately 80 people present at the talk, about 50
carried protest signs. The protestors (sic) did not try to interfere
with the seminar. They simply raised their signs as Pike began
to speak. This accomplished the purpose of making their ire known.

---

I accomplished my purpose of delivering a ``seminar on [my] latest
research project.'' Other than being a good talk, it was also,
in the end, pretty ordinary.